[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:37 dist-test-slave-5bn2 krb5kdc[18956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:37 dist-test-slave-5bn2 krb5kdc[18956](info): set up 2 sockets
Aug 07 15:06:37 dist-test-slave-5bn2 krb5kdc[18956](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 15:06:38.497309 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:38 dist-test-slave-5bn2 krb5kdc[18956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:38.619462 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:33019
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:33019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:38.703624 18975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:38.703843 18975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:38.703876 18975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:38.705013 18975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:38.705052 18975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:38.705070 18975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:38.705085 18975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:38.705101 18975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:38.706792 18975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:33019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:33019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:38.707065 18975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:38.707274 18975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:38.712478 18975 server_base.cc:1034] running on GCE node
W20260807 15:06:38.712553 18981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:38.712622 18984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:38.712594 18982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:38.713017 18975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:38.713399 18975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:38.714550 18975 hybrid_clock.cc:648] HybridClock initialized: now 1786115198714551 us; error 61 us; skew 500 ppm
Aug 07 15:06:38 dist-test-slave-5bn2 krb5kdc[18956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115198, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:38.717042 18975 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:38.717481 18975 webserver.cc:460] Webserver started at http://127.18.129.62:36473/ using document root <none> and password file <none>
I20260807 15:06:38.717669 18975 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:38.717706 18975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:38.717794 18975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:38.718906 18975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "90cacf03b83d4a13980b1396ce8a0e65"
format_stamp: "Formatted at 2026-08-07 15:06:38 on dist-test-slave-5bn2"
server_key: "7630ede3dd6c9394fda654cd133715e3"
server_key_iv: "a26b08d93c6dd4cf5ecfb802de48f98b"
server_key_version: "encryptionkey@0"
I20260807 15:06:38.719223 18975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "90cacf03b83d4a13980b1396ce8a0e65"
format_stamp: "Formatted at 2026-08-07 15:06:38 on dist-test-slave-5bn2"
server_key: "7630ede3dd6c9394fda654cd133715e3"
server_key_iv: "a26b08d93c6dd4cf5ecfb802de48f98b"
server_key_version: "encryptionkey@0"
I20260807 15:06:38.720604 18975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:38.721269 18997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:38.721493 18975 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:06:38.721593 18975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "90cacf03b83d4a13980b1396ce8a0e65"
format_stamp: "Formatted at 2026-08-07 15:06:38 on dist-test-slave-5bn2"
server_key: "7630ede3dd6c9394fda654cd133715e3"
server_key_iv: "a26b08d93c6dd4cf5ecfb802de48f98b"
server_key_version: "encryptionkey@0"
I20260807 15:06:38.721666 18975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:38.746146 18975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:38.748008 18975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:38.748132 18975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:38.752455 18975 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:33019
I20260807 15:06:38.752501 19069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:33019 every 8 connection(s)
I20260807 15:06:38.752890 18975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:38.753823 19070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:38.754245 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 18975
I20260807 15:06:38.754326 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:38.754951 18948 external_mini_cluster.cc:1428] Setting key 5c1ac7c9f746b9bed78c7ee7391d3fc9
I20260807 15:06:38.756484 19070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65: Bootstrap starting.
I20260807 15:06:38.757036 19070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:38.757303 19070 log.cc:824] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:38.757997 19070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65: No bootstrap required, opened a new log
I20260807 15:06:38.759716 19070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90cacf03b83d4a13980b1396ce8a0e65" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 33019 } }
I20260807 15:06:38.759851 19070 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:38.759873 19070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90cacf03b83d4a13980b1396ce8a0e65, State: Initialized, Role: FOLLOWER
I20260807 15:06:38.759980 19070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90cacf03b83d4a13980b1396ce8a0e65" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 33019 } }
I20260807 15:06:38.760102 19070 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:38.760134 19070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:38.760183 19070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:38.760906 19070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90cacf03b83d4a13980b1396ce8a0e65" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 33019 } }
I20260807 15:06:38.761016 19070 leader_election.cc:302] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90cacf03b83d4a13980b1396ce8a0e65; no voters:
I20260807 15:06:38.761183 19070 leader_election.cc:288] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:38.761230 19078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:38.761379 19078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [term 1 LEADER]: Becoming Leader. State: Replica: 90cacf03b83d4a13980b1396ce8a0e65, State: Running, Role: LEADER
I20260807 15:06:38.761425 19070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:38.761468 19078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90cacf03b83d4a13980b1396ce8a0e65" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 33019 } }
I20260807 15:06:38.761809 19078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90cacf03b83d4a13980b1396ce8a0e65. Latest consensus state: current_term: 1 leader_uuid: "90cacf03b83d4a13980b1396ce8a0e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90cacf03b83d4a13980b1396ce8a0e65" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 33019 } } }
I20260807 15:06:38.761885 19078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:38.762097 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90cacf03b83d4a13980b1396ce8a0e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90cacf03b83d4a13980b1396ce8a0e65" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 33019 } } }
I20260807 15:06:38.762159 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:38.762451 19088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:38.762998 19088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:38.764550 19088 catalog_manager.cc:1347] Generated new cluster ID: 46cfdda571f740459991b56e7bbda075
I20260807 15:06:38.764608 19088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:06:38 dist-test-slave-5bn2 krb5kdc[18956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:38.772758 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:38.755848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.18.129.62:33019)
0807 15:06:38.756019 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:38.756028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:38.756224 (+ 196us) server_negotiation.cc:407] Connection header received
0807 15:06:38.758375 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:38.758384 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:38.758444 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:38.758592 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:38.760180 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:38.762086 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:38.763795 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:38.764349 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:38.765437 (+ 1088us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:38.768639 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:38.768644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:38.768644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:38.768667 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:38.770562 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:38.771098 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:38.771101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:38.771102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:38.771202 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:38.771476 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:38.771477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:38.771477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:38.771757 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:38.771899 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:38.772180 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 15:06:38.772310 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260807 15:06:38.782656 19088 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:38.783212 19088 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:38.789404 19088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90cacf03b83d4a13980b1396ce8a0e65: Generated new TSK 0
I20260807 15:06:38.789556 19088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:06:38.816030 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:33019
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:38.897336 19111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:38.897540 19111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:38.897572 19111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:38.898685 19111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:38.898727 19111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:38.898762 19111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:38.900241 19111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:33019
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:38.900488 19111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:38.900710 19111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:38.905721 19116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:38.905755 19120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:38.905889 19111 server_base.cc:1034] running on GCE node
W20260807 15:06:38.905784 19118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:38.906136 19111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:38.906374 19111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:38.907509 19111 hybrid_clock.cc:648] HybridClock initialized: now 1786115198907496 us; error 30 us; skew 500 ppm
Aug 07 15:06:38 dist-test-slave-5bn2 krb5kdc[18956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115198, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:38.909922 19111 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:06:38.910342 19111 webserver.cc:460] Webserver started at http://127.18.129.1:40977/ using document root <none> and password file <none>
I20260807 15:06:38.910557 19111 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:38.910611 19111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:38.910737 19111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:38.911828 19111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "32e7bed2a7cc408b8ab56e29f194196d"
format_stamp: "Formatted at 2026-08-07 15:06:38 on dist-test-slave-5bn2"
server_key: "9dff0e8271682087efd6829958cb665f"
server_key_iv: "a2ecb86591c41d0b9c4fce15cab572e0"
server_key_version: "encryptionkey@0"
I20260807 15:06:38.912165 19111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "32e7bed2a7cc408b8ab56e29f194196d"
format_stamp: "Formatted at 2026-08-07 15:06:38 on dist-test-slave-5bn2"
server_key: "9dff0e8271682087efd6829958cb665f"
server_key_iv: "a2ecb86591c41d0b9c4fce15cab572e0"
server_key_version: "encryptionkey@0"
I20260807 15:06:38.913393 19111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:06:38.914695 19127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:38.914875 19111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:38.914971 19111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "32e7bed2a7cc408b8ab56e29f194196d"
format_stamp: "Formatted at 2026-08-07 15:06:38 on dist-test-slave-5bn2"
server_key: "9dff0e8271682087efd6829958cb665f"
server_key_iv: "a2ecb86591c41d0b9c4fce15cab572e0"
server_key_version: "encryptionkey@0"
I20260807 15:06:38.915040 19111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:38.933951 19111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:38.935701 19111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:38.935840 19111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:38.936060 19111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:38.936362 19111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:38.936398 19111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:38.936427 19111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:38.936450 19111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:38.943519 19111 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:43037
I20260807 15:06:38.943584 19286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:43037 every 8 connection(s)
I20260807 15:06:38.943887 19111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:06:38 dist-test-slave-5bn2 krb5kdc[18956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115198, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:38.951025 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19111
I20260807 15:06:38.951128 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:38.951295 18948 external_mini_cluster.cc:1428] Setting key b7d524a85b420aadc5fca8b372e14c75
I20260807 15:06:38.952900 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:38.944563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:41195 (local address 127.18.129.62:33019)
0807 15:06:38.944627 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:38.944629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:38.944864 (+ 235us) server_negotiation.cc:407] Connection header received
0807 15:06:38.945379 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:38.945381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:38.945393 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:38.945445 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:38.946490 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:38.947061 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:38.948032 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:38.948244 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:38.948300 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:38.950322 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:38.950324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:38.950324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:38.950332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:38.951668 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:38.952024 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:38.952026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:38.952026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:38.952089 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:38.952303 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:38.952304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:38.952304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:38.952413 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:38.952468 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:38.952726 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 15:06:38.952811 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:06:38.953063 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:38.944523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:33019 (local address 127.18.129.1:41195)
0807 15:06:38.944750 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:38.944786 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:38.945280 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:38.945514 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:38.945518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:38.946188 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:38.946387 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:38.946393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:38.947147 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:38.947148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:38.947946 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:38.947949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:38.948316 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:38.948317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:38.948437 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:38.948872 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:38.948880 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:38.950238 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:38.951734 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:38.951742 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:38.951744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:38.951958 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:38.952143 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:38.952144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:38.952144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:38.952248 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:38.952466 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:38.952473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:38.952610 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 15:06:38.952697 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:06:38.952850 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":161,"threads_started":1}
I20260807 15:06:38.953490 19287 heartbeater.cc:347] Connected to a master server at 127.18.129.62:33019
I20260807 15:06:38.953585 19287 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:38.953758 19287 heartbeater.cc:511] Master 127.18.129.62:33019 requested a full tablet report, sending...
I20260807 15:06:38.954258 19020 ts_manager.cc:194] Registered new tserver with Master: 32e7bed2a7cc408b8ab56e29f194196d (127.18.129.1:43037)
I20260807 15:06:38.954969 19020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:41195
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:06:38.975845 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:33019
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:39.059059 19292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:39.059279 19292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:39.059305 19292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:39.060433 19292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:39.060480 19292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:39.060508 19292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:39.062059 19292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:33019
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:39.062340 19292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:39.062558 19292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:39.067672 19300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:39.067687 19302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:39.067687 19299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:39.067756 19292 server_base.cc:1034] running on GCE node
I20260807 15:06:39.068092 19292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:39.068359 19292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:39.069547 19292 hybrid_clock.cc:648] HybridClock initialized: now 1786115199069515 us; error 31 us; skew 500 ppm
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[18956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115199, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:39.072095 19292 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:06:39.072490 19292 webserver.cc:460] Webserver started at http://127.18.129.2:45733/ using document root <none> and password file <none>
I20260807 15:06:39.072710 19292 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:39.072758 19292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:39.072883 19292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:39.073918 19292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "1e7acd2680eb47719995bec99eeb84c3"
format_stamp: "Formatted at 2026-08-07 15:06:39 on dist-test-slave-5bn2"
server_key: "4144b2170595536e7ace31329b31cca8"
server_key_iv: "12408436b8917131500c81ef0483e98b"
server_key_version: "encryptionkey@0"
I20260807 15:06:39.074266 19292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "1e7acd2680eb47719995bec99eeb84c3"
format_stamp: "Formatted at 2026-08-07 15:06:39 on dist-test-slave-5bn2"
server_key: "4144b2170595536e7ace31329b31cca8"
server_key_iv: "12408436b8917131500c81ef0483e98b"
server_key_version: "encryptionkey@0"
I20260807 15:06:39.075569 19292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:39.076227 19314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:39.076380 19292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:39.076468 19292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "1e7acd2680eb47719995bec99eeb84c3"
format_stamp: "Formatted at 2026-08-07 15:06:39 on dist-test-slave-5bn2"
server_key: "4144b2170595536e7ace31329b31cca8"
server_key_iv: "12408436b8917131500c81ef0483e98b"
server_key_version: "encryptionkey@0"
I20260807 15:06:39.076529 19292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:39.097389 19292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:39.099121 19292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:39.099268 19292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:39.099491 19292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:39.099784 19292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:39.099819 19292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:39.099849 19292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:39.099867 19292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:39.106832 19292 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:36751
I20260807 15:06:39.106892 19479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:36751 every 8 connection(s)
I20260807 15:06:39.107226 19292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:06:39.110772 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19292
I20260807 15:06:39.110900 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:39.111052 18948 external_mini_cluster.cc:1428] Setting key 6b6e983d2fbf794450e41b18b11be682
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[18956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115199, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:39.116508 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:39.108031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:53599 (local address 127.18.129.62:33019)
0807 15:06:39.108089 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:39.108091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:39.108278 (+ 187us) server_negotiation.cc:407] Connection header received
0807 15:06:39.108792 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:39.108793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:39.108806 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:39.108873 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:39.109795 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.110323 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.111504 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.111751 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.111810 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:39.113827 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:39.113829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:39.113829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:39.113836 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:39.115245 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:39.115623 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:39.115625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:39.115625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:39.115682 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:39.115900 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:39.115901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:39.115901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:39.116014 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:39.116078 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:39.116333 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 15:06:39.116416 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:39.116727 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:39.108008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:33019 (local address 127.18.129.2:53599)
0807 15:06:39.108163 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:39.108200 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:39.108697 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:39.108930 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:39.108933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:39.109496 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:39.109694 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:39.109699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.110396 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.110397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:39.111427 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:39.111430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.111826 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.111827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:39.111959 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:39.112421 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:39.112430 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:39.113740 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:39.115320 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:39.115327 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:39.115329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:39.115555 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:39.115737 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:39.115738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:39.115738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:39.115845 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:39.116075 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:39.116082 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:39.116217 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 15:06:39.116304 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:06:39.116455 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260807 15:06:39.117167 19480 heartbeater.cc:347] Connected to a master server at 127.18.129.62:33019
I20260807 15:06:39.117276 19480 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:39.117501 19480 heartbeater.cc:511] Master 127.18.129.62:33019 requested a full tablet report, sending...
I20260807 15:06:39.117949 19020 ts_manager.cc:194] Registered new tserver with Master: 1e7acd2680eb47719995bec99eeb84c3 (127.18.129.2:36751)
I20260807 15:06:39.118419 19020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:53599
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:06:39.137876 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:33019
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:39.218945 19488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:39.219163 19488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:39.219192 19488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:39.220304 19488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:39.220337 19488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:39.220362 19488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:39.221830 19488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:33019
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:39.222023 19488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:39.222213 19488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:39.227332 19488 server_base.cc:1034] running on GCE node
W20260807 15:06:39.227308 19495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:39.227324 19498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:39.227308 19496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:39.227655 19488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:39.227886 19488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:39.229027 19488 hybrid_clock.cc:648] HybridClock initialized: now 1786115199229006 us; error 30 us; skew 500 ppm
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[18956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115199, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:39.231506 19488 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:06:39.232016 19488 webserver.cc:460] Webserver started at http://127.18.129.3:37137/ using document root <none> and password file <none>
I20260807 15:06:39.232254 19488 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:39.232303 19488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:39.232430 19488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:39.233479 19488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "9a260b2954604c7880094f130e97949b"
format_stamp: "Formatted at 2026-08-07 15:06:39 on dist-test-slave-5bn2"
server_key: "b5be96670c280ea9b71f7d1a17527123"
server_key_iv: "0e11e25423dd3b749c0b9ca02692371a"
server_key_version: "encryptionkey@0"
I20260807 15:06:39.233811 19488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "9a260b2954604c7880094f130e97949b"
format_stamp: "Formatted at 2026-08-07 15:06:39 on dist-test-slave-5bn2"
server_key: "b5be96670c280ea9b71f7d1a17527123"
server_key_iv: "0e11e25423dd3b749c0b9ca02692371a"
server_key_version: "encryptionkey@0"
I20260807 15:06:39.235140 19488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:39.235930 19507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:39.236123 19488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:39.236217 19488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "9a260b2954604c7880094f130e97949b"
format_stamp: "Formatted at 2026-08-07 15:06:39 on dist-test-slave-5bn2"
server_key: "b5be96670c280ea9b71f7d1a17527123"
server_key_iv: "0e11e25423dd3b749c0b9ca02692371a"
server_key_version: "encryptionkey@0"
I20260807 15:06:39.236281 19488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:39.259639 19488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:39.262466 19488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:39.262704 19488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:39.263008 19488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:39.263382 19488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:39.263422 19488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:39.263454 19488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:39.263478 19488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:39.271974 19488 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:36007
I20260807 15:06:39.272040 19663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:36007 every 8 connection(s)
I20260807 15:06:39.272348 19488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:06:39.273252 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19488
I20260807 15:06:39.273335 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:39.273492 18948 external_mini_cluster.cc:1428] Setting key 9f94bc4d260224839d3557303d785b09
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[18956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115199, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:39.281564 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:39.273393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:39777 (local address 127.18.129.62:33019)
0807 15:06:39.273456 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:39.273458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:39.273469 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:06:39.274004 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:39.274005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:39.274018 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:39.274080 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:39.275228 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.275928 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.276882 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.277075 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.277135 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:39.279201 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:39.279203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:39.279203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:39.279210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:39.280511 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:39.280851 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:39.280852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:39.280853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:39.280900 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:39.281086 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:39.281086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:39.281086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:39.281184 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:39.281230 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:39.281431 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 15:06:39.281487 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:06:39.281752 19666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:39.273136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:33019 (local address 127.18.129.3:39777)
0807 15:06:39.273307 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:39.273346 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:39.273918 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:39.274155 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:39.274160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:39.274869 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:39.275086 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:39.275092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.276013 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.276014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:39.276805 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:39.276808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.277141 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.277142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:39.277243 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:39.277703 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:39.277712 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:39.279122 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:39.280582 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:39.280588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:39.280590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:39.280796 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:39.280946 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:39.280947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:39.280947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:39.281042 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:39.281227 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:39.281231 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:39.281343 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 15:06:39.281416 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:06:39.281533 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260807 15:06:39.282214 19664 heartbeater.cc:347] Connected to a master server at 127.18.129.62:33019
I20260807 15:06:39.282298 19664 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:39.282464 19664 heartbeater.cc:511] Master 127.18.129.62:33019 requested a full tablet report, sending...
I20260807 15:06:39.282819 19020 ts_manager.cc:194] Registered new tserver with Master: 9a260b2954604c7880094f130e97949b (127.18.129.3:36007)
I20260807 15:06:39.283203 19020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:39777
I20260807 15:06:39.285373 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[18956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[18956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:06:39.308904 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:39.301612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.18.129.1:43037)
0807 15:06:39.301785 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:39.301791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:39.301801 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:39.301827 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:39.301829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:39.301841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:39.301906 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:39.302454 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.303254 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.303951 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.304273 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.304334 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:39.306431 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:39.306441 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:39.306442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:39.306450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:39.307943 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:39.308261 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:39.308264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:39.308264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:39.308319 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:39.308446 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:39.308446 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:39.308446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:39.308593 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:39.308723 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:39.308757 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:06:39.308821 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
W20260807 15:06:39.309352 19160 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38020
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[18956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[18956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:06:39.332647 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:39.326372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38032 (local address 127.18.129.1:43037)
0807 15:06:39.326445 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:39.326447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:39.326517 (+ 70us) server_negotiation.cc:407] Connection header received
0807 15:06:39.326563 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:39.326564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:39.326577 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:39.326625 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:39.327216 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.327712 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.328364 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:39.328557 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:39.328599 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:39.330578 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:39.330579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:39.330579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:39.330586 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:39.331860 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:39.332114 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:39.332115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:39.332116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:39.332161 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:39.332276 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:39.332276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:39.332276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:39.332358 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:39.332435 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:39.332491 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:06:39.332563 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:06:39.333194 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19111
I20260807 15:06:39.345367 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19292
I20260807 15:06:39.371492 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19488
I20260807 15:06:39.379508 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 18975
2026-08-07T15:06:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2039 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[19682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[19682](info): set up 2 sockets
Aug 07 15:06:39 dist-test-slave-5bn2 krb5kdc[19682](info): commencing operation
krb5kdc: starting...
W20260807 15:06:40.509068 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:40 dist-test-slave-5bn2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:40.625789 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34557
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:34557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:40.706378 19700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:40.706593 19700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:40.706622 19700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:40.707780 19700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:40.707825 19700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:40.707844 19700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:40.707859 19700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:40.707875 19700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:40.709350 19700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:34557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:40.709589 19700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:40.709792 19700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:40.714730 19700 server_base.cc:1034] running on GCE node
W20260807 15:06:40.714702 19709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:40.714660 19707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:40.714675 19713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:40.715154 19700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:40.715421 19700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:40.716557 19700 hybrid_clock.cc:648] HybridClock initialized: now 1786115200716543 us; error 29 us; skew 500 ppm
Aug 07 15:06:40 dist-test-slave-5bn2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115200, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:40.718989 19700 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:40.719444 19700 webserver.cc:460] Webserver started at http://127.18.129.62:35023/ using document root <none> and password file <none>
I20260807 15:06:40.719636 19700 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:40.719679 19700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:40.719795 19700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:40.720835 19700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "bb263acfd5cf45ebb607976cce4a5257"
format_stamp: "Formatted at 2026-08-07 15:06:40 on dist-test-slave-5bn2"
server_key: "cbfbaa3612620fdadc328c132d47cc0f"
server_key_iv: "13e99999256ffc68756fdea949d71c5e"
server_key_version: "encryptionkey@0"
I20260807 15:06:40.721167 19700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "bb263acfd5cf45ebb607976cce4a5257"
format_stamp: "Formatted at 2026-08-07 15:06:40 on dist-test-slave-5bn2"
server_key: "cbfbaa3612620fdadc328c132d47cc0f"
server_key_iv: "13e99999256ffc68756fdea949d71c5e"
server_key_version: "encryptionkey@0"
I20260807 15:06:40.722427 19700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:40.723098 19720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:40.723229 19700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:40.723312 19700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "bb263acfd5cf45ebb607976cce4a5257"
format_stamp: "Formatted at 2026-08-07 15:06:40 on dist-test-slave-5bn2"
server_key: "cbfbaa3612620fdadc328c132d47cc0f"
server_key_iv: "13e99999256ffc68756fdea949d71c5e"
server_key_version: "encryptionkey@0"
I20260807 15:06:40.723377 19700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:40.738816 19700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:40.740940 19700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:40.741081 19700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:40.744910 19700 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:34557
I20260807 15:06:40.744956 19795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:34557 every 8 connection(s)
I20260807 15:06:40.745278 19700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:40.746097 19796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:40.748332 19796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257: Bootstrap starting.
I20260807 15:06:40.748855 19796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:40.749094 19796 log.cc:824] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:40.749709 19796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257: No bootstrap required, opened a new log
I20260807 15:06:40.750525 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19700
I20260807 15:06:40.750613 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:40.750775 18948 external_mini_cluster.cc:1428] Setting key e1d1801c384825f0f618a639076de625
I20260807 15:06:40.751695 19796 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb263acfd5cf45ebb607976cce4a5257" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34557 } }
I20260807 15:06:40.751852 19796 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:40.751889 19796 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb263acfd5cf45ebb607976cce4a5257, State: Initialized, Role: FOLLOWER
I20260807 15:06:40.752019 19796 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb263acfd5cf45ebb607976cce4a5257" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34557 } }
I20260807 15:06:40.752100 19796 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:40.752139 19796 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:40.752188 19796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:40.753222 19796 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb263acfd5cf45ebb607976cce4a5257" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34557 } }
I20260807 15:06:40.753314 19796 leader_election.cc:302] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb263acfd5cf45ebb607976cce4a5257; no voters:
I20260807 15:06:40.753458 19796 leader_election.cc:288] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:40.753500 19801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:40.753649 19801 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [term 1 LEADER]: Becoming Leader. State: Replica: bb263acfd5cf45ebb607976cce4a5257, State: Running, Role: LEADER
I20260807 15:06:40.753710 19796 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:40.753760 19801 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb263acfd5cf45ebb607976cce4a5257" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34557 } }
I20260807 15:06:40.754164 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb263acfd5cf45ebb607976cce4a5257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb263acfd5cf45ebb607976cce4a5257" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34557 } } }
I20260807 15:06:40.754175 19803 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb263acfd5cf45ebb607976cce4a5257. Latest consensus state: current_term: 1 leader_uuid: "bb263acfd5cf45ebb607976cce4a5257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb263acfd5cf45ebb607976cce4a5257" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34557 } } }
I20260807 15:06:40.754262 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:40.754278 19803 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:40.755043 19813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:40.755550 19813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:06:40 dist-test-slave-5bn2 krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:40.758250 19813 catalog_manager.cc:1347] Generated new cluster ID: 29c1f872e2df4b2ea6c75835d56b0441
I20260807 15:06:40.758303 19813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:40.761106 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:40.751309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.18.129.62:34557)
0807 15:06:40.751469 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:40.751476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:40.751502 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:06:40.751653 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:40.751658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:40.751699 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:40.751801 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:40.752547 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:40.753498 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:40.754525 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:40.755130 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:40.755237 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:40.757627 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:40.757631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:40.757632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:40.757645 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:40.759554 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:40.759947 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:40.759949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:40.759950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:40.760076 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:40.760266 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:40.760267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:40.760267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:40.760531 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:40.760690 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:40.760774 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:06:40.760901 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260807 15:06:40.781684 19813 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:40.782223 19813 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:40.784163 19813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb263acfd5cf45ebb607976cce4a5257: Generated new TSK 0
I20260807 15:06:40.784315 19813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:06:40.814286 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34557
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:40.894539 19826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:40.894760 19826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:40.894783 19826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:40.895932 19826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:40.895977 19826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:40.896005 19826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:40.897538 19826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34557
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:40.897784 19826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:40.897993 19826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:40.903048 19826 server_base.cc:1034] running on GCE node
W20260807 15:06:40.903031 19832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:40.903025 19835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:40.903030 19833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:40.903498 19826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:40.903738 19826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:40.904891 19826 hybrid_clock.cc:648] HybridClock initialized: now 1786115200904877 us; error 28 us; skew 500 ppm
Aug 07 15:06:40 dist-test-slave-5bn2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115200, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:40.907369 19826 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:06:40.907781 19826 webserver.cc:460] Webserver started at http://127.18.129.1:42777/ using document root <none> and password file <none>
I20260807 15:06:40.907989 19826 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:40.908031 19826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:40.908164 19826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:40.909246 19826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "56f0e32835954741825b21534b50ebe4"
format_stamp: "Formatted at 2026-08-07 15:06:40 on dist-test-slave-5bn2"
server_key: "43bf56fdf02ebef52f9ece495fa0249d"
server_key_iv: "b92ad26862549edc486ff0fe55524556"
server_key_version: "encryptionkey@0"
I20260807 15:06:40.909581 19826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "56f0e32835954741825b21534b50ebe4"
format_stamp: "Formatted at 2026-08-07 15:06:40 on dist-test-slave-5bn2"
server_key: "43bf56fdf02ebef52f9ece495fa0249d"
server_key_iv: "b92ad26862549edc486ff0fe55524556"
server_key_version: "encryptionkey@0"
I20260807 15:06:40.910863 19826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:40.911619 19846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:40.911791 19826 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:40.911881 19826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "56f0e32835954741825b21534b50ebe4"
format_stamp: "Formatted at 2026-08-07 15:06:40 on dist-test-slave-5bn2"
server_key: "43bf56fdf02ebef52f9ece495fa0249d"
server_key_iv: "b92ad26862549edc486ff0fe55524556"
server_key_version: "encryptionkey@0"
I20260807 15:06:40.911938 19826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:40.924113 19826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:40.926297 19826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:40.926429 19826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:40.926635 19826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:40.926967 19826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:40.927023 19826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:40.927062 19826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:40.927084 19826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:40.932614 19826 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:45933
I20260807 15:06:40.932687 20007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:45933 every 8 connection(s)
I20260807 15:06:40.932968 19826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:06:40 dist-test-slave-5bn2 krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115200, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:40.939189 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19826
I20260807 15:06:40.939278 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:40.939429 18948 external_mini_cluster.cc:1428] Setting key 69957cd7da0494df05b4e463758a0eb7
I20260807 15:06:40.942239 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:40.933645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:43505 (local address 127.18.129.62:34557)
0807 15:06:40.933698 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:40.933700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:40.933885 (+ 185us) server_negotiation.cc:407] Connection header received
0807 15:06:40.935124 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:40.935125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:40.935140 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:40.935199 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:40.935773 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:40.936270 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:40.937208 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:40.937398 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:40.937441 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:40.939534 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:40.939535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:40.939535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:40.939542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:40.940949 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:40.941321 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:40.941322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:40.941322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:40.941386 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:40.941592 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:40.941593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:40.941593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:40.941726 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:40.941792 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:40.942106 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 15:06:40.942174 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:06:40.942459 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:40.933612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34557 (local address 127.18.129.1:43505)
0807 15:06:40.933763 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:40.933805 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:40.935024 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:40.935257 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:40.935260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:40.935510 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:40.935708 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:40.935713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:40.936337 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:40.936337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:40.937145 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:40.937147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:40.937452 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:40.937452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:40.937556 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:40.938027 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:40.938035 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:40.939431 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:40.941027 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:40.941035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:40.941038 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:40.941253 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:40.941441 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:40.941441 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:40.941442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:40.941542 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:40.941790 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:40.941797 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:40.941965 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 15:06:40.942074 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 15:06:40.942238 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260807 15:06:40.942927 20008 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34557
I20260807 15:06:40.943025 20008 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:40.943198 20008 heartbeater.cc:511] Master 127.18.129.62:34557 requested a full tablet report, sending...
I20260807 15:06:40.943701 19743 ts_manager.cc:194] Registered new tserver with Master: 56f0e32835954741825b21534b50ebe4 (127.18.129.1:45933)
I20260807 15:06:40.944339 19743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:43505
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:06:40.963686 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34557
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:41.044251 20015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:41.044461 20015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:41.044493 20015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:41.045616 20015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:41.045658 20015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:41.045693 20015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:41.047174 20015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34557
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:41.047403 20015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:41.047605 20015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:41.052445 20024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:41.052546 20025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:41.052650 20015 server_base.cc:1034] running on GCE node
W20260807 15:06:41.052455 20027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:41.052930 20015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:41.053158 20015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:41.054288 20015 hybrid_clock.cc:648] HybridClock initialized: now 1786115201054279 us; error 27 us; skew 500 ppm
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115201, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:41.056752 20015 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:06:41.057144 20015 webserver.cc:460] Webserver started at http://127.18.129.2:42391/ using document root <none> and password file <none>
I20260807 15:06:41.057317 20015 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:41.057354 20015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:41.057442 20015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:41.058452 20015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "aa0e532b7d2e424f81d29cc734a99c81"
format_stamp: "Formatted at 2026-08-07 15:06:41 on dist-test-slave-5bn2"
server_key: "f76f08b2ff02a9fb6b240dbc95c6929b"
server_key_iv: "5749bfece0ef141f13019f458d849095"
server_key_version: "encryptionkey@0"
I20260807 15:06:41.058751 20015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "aa0e532b7d2e424f81d29cc734a99c81"
format_stamp: "Formatted at 2026-08-07 15:06:41 on dist-test-slave-5bn2"
server_key: "f76f08b2ff02a9fb6b240dbc95c6929b"
server_key_iv: "5749bfece0ef141f13019f458d849095"
server_key_version: "encryptionkey@0"
I20260807 15:06:41.059962 20015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:41.060609 20037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:41.060767 20015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:06:41.060850 20015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "aa0e532b7d2e424f81d29cc734a99c81"
format_stamp: "Formatted at 2026-08-07 15:06:41 on dist-test-slave-5bn2"
server_key: "f76f08b2ff02a9fb6b240dbc95c6929b"
server_key_iv: "5749bfece0ef141f13019f458d849095"
server_key_version: "encryptionkey@0"
I20260807 15:06:41.060905 20015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:41.092332 20015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:41.094120 20015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:41.094241 20015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:41.094445 20015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:41.094712 20015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:41.094743 20015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:41.094765 20015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:41.094782 20015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:41.100562 20015 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:37563
I20260807 15:06:41.100649 20201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:37563 every 8 connection(s)
I20260807 15:06:41.100915 20015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115201, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:41.108464 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20015
I20260807 15:06:41.108566 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:41.108741 18948 external_mini_cluster.cc:1428] Setting key dd452298d52883d1410e2796bfecb8b1
I20260807 15:06:41.110363 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.101656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:33843 (local address 127.18.129.62:34557)
0807 15:06:41.101711 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.101713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.101941 (+ 228us) server_negotiation.cc:407] Connection header received
0807 15:06:41.102937 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.102938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.102947 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.102992 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:41.103608 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.104101 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.105148 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.105332 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.105378 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.107642 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:41.107644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:41.107644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:41.107651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:41.109009 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.109454 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.109455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.109456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.109549 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.109769 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.109770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.109771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.109896 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:41.109959 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.110186 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.110273 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:06:41.110443 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.101629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34557 (local address 127.18.129.2:33843)
0807 15:06:41.101815 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:41.101858 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:41.102800 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:41.103053 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:41.103056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:41.103333 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:41.103538 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.103543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.104187 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.104188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.105075 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.105078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.105449 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.105449 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.105554 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.106079 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:41.106087 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:41.107558 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:41.109085 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:41.109094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:41.109096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:41.109385 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:41.109608 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:41.109609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:41.109609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:41.109712 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:41.109951 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:41.109954 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:41.110061 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 15:06:41.110151 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:06:41.110272 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260807 15:06:41.110895 20202 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34557
I20260807 15:06:41.110994 20202 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:41.111174 20202 heartbeater.cc:511] Master 127.18.129.62:34557 requested a full tablet report, sending...
I20260807 15:06:41.111588 19743 ts_manager.cc:194] Registered new tserver with Master: aa0e532b7d2e424f81d29cc734a99c81 (127.18.129.2:37563)
I20260807 15:06:41.111934 19743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:33843
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:06:41.133934 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34557
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:41.214705 20207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:41.214964 20207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:41.214989 20207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:41.216118 20207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:41.216152 20207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:41.216176 20207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:41.217595 20207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34557
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:41.217793 20207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:41.217969 20207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:41.223075 20213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:41.223065 20214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:41.223065 20216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:41.223307 20207 server_base.cc:1034] running on GCE node
I20260807 15:06:41.223658 20207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:41.223866 20207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:41.225003 20207 hybrid_clock.cc:648] HybridClock initialized: now 1786115201224987 us; error 37 us; skew 500 ppm
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115201, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:41.227454 20207 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:06:41.227901 20207 webserver.cc:460] Webserver started at http://127.18.129.3:35635/ using document root <none> and password file <none>
I20260807 15:06:41.228119 20207 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:41.228169 20207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:41.228294 20207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:41.229344 20207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "8ea21254339946f199218d124d20eb14"
format_stamp: "Formatted at 2026-08-07 15:06:41 on dist-test-slave-5bn2"
server_key: "15776ab4929a704bc5da0fe237f74644"
server_key_iv: "da534e2a9b6d8c0e64f5e24a5bea4c7c"
server_key_version: "encryptionkey@0"
I20260807 15:06:41.229691 20207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "8ea21254339946f199218d124d20eb14"
format_stamp: "Formatted at 2026-08-07 15:06:41 on dist-test-slave-5bn2"
server_key: "15776ab4929a704bc5da0fe237f74644"
server_key_iv: "da534e2a9b6d8c0e64f5e24a5bea4c7c"
server_key_version: "encryptionkey@0"
I20260807 15:06:41.230986 20207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:41.231711 20231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:41.231926 20207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:41.232026 20207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "8ea21254339946f199218d124d20eb14"
format_stamp: "Formatted at 2026-08-07 15:06:41 on dist-test-slave-5bn2"
server_key: "15776ab4929a704bc5da0fe237f74644"
server_key_iv: "da534e2a9b6d8c0e64f5e24a5bea4c7c"
server_key_version: "encryptionkey@0"
I20260807 15:06:41.232091 20207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:41.251670 20207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:41.253468 20207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:41.253633 20207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:41.253862 20207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:41.254190 20207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:41.254236 20207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:41.254268 20207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:41.254292 20207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:41.260083 20207 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:33251
I20260807 15:06:41.260159 20390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:33251 every 8 connection(s)
I20260807 15:06:41.260437 20207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115201, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:41.268925 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20207
I20260807 15:06:41.269029 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:41.269193 18948 external_mini_cluster.cc:1428] Setting key 3f5d409eb8b05a61eff025c81ddd6c6e
I20260807 15:06:41.270323 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.261109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:48581 (local address 127.18.129.62:34557)
0807 15:06:41.261170 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.261172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.261376 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:06:41.262285 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.262286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.262297 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.262357 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:41.263000 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.263579 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.264562 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.264779 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.264832 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.266878 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:41.266880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:41.266880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:41.266888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:41.268605 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.269117 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.269119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.269119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.269190 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.269492 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.269493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.269494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.269649 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:41.269719 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.270069 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.270207 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:06:41.270470 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.261066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34557 (local address 127.18.129.3:48581)
0807 15:06:41.261251 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:41.261293 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:41.262184 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:41.262424 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:41.262427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:41.262692 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:41.262923 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.262929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.263648 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.263649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.264480 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.264484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.264851 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.264852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.264977 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.265440 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:41.265449 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:41.266768 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:41.268703 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:41.268712 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:41.268715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:41.269018 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:41.269263 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:41.269263 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:41.269264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:41.269412 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:41.269711 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:41.269718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:41.269882 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 15:06:41.270030 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 15:06:41.270226 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260807 15:06:41.271075 20391 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34557
I20260807 15:06:41.271173 20391 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:41.271341 20391 heartbeater.cc:511] Master 127.18.129.62:34557 requested a full tablet report, sending...
I20260807 15:06:41.271723 19743 ts_manager.cc:194] Registered new tserver with Master: 8ea21254339946f199218d124d20eb14 (127.18.129.3:33251)
I20260807 15:06:41.272092 19743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:48581
I20260807 15:06:41.280933 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:06:41.288383 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.282793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60188 (local address 127.18.129.62:34557)
0807 15:06:41.282878 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.282880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.282966 (+ 86us) server_negotiation.cc:407] Connection header received
0807 15:06:41.283032 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.283033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.283043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.283101 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:41.283639 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.284219 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.284906 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.285101 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.285151 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.286193 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:41.286194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:41.286194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:41.286201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:41.287491 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.287786 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.287786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.287787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.287834 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.287959 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.287959 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.287959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.288104 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:41.288174 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.288262 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.288318 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:06:41.290943 19743 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:41.292740 19743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:41.299512 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.295710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:37563 (local address 127.0.0.1:42410)
0807 15:06:41.296088 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:41.296097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:41.296153 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:41.296429 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:41.296429 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:41.296432 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:41.296475 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.296477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.297662 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.297663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.298425 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.298427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.299219 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.299220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.299301 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.299315 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:06:41.299381 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:06:41.299437 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":63,"threads_started":1}
I20260807 15:06:41.299548 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.295838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.18.129.2:37563)
0807 15:06:41.295978 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.295981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.296208 (+ 227us) server_negotiation.cc:407] Connection header received
0807 15:06:41.296252 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.296255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.296269 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.296334 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:41.296553 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.297576 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.298506 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.299142 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.299206 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.299267 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.299432 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.299476 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260807 15:06:41.299858 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.295494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:33251 (local address 127.0.0.1:42512)
0807 15:06:41.295855 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:41.295906 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:41.295976 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:41.296200 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:41.296203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:41.296204 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:41.296278 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.296281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.297576 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.297577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.298618 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.298622 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.299633 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.299634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.299698 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.299701 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:41.299746 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:06:41.299785 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":62,"threads_started":1}
I20260807 15:06:41.300132 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.295675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34452 (local address 127.18.129.1:45933)
0807 15:06:41.296041 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.296045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.296146 (+ 101us) server_negotiation.cc:407] Connection header received
0807 15:06:41.296196 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.296199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.296215 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.296295 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:41.296380 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.297262 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.298745 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.299481 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.299531 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.299794 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.300001 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.300049 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":98,"threads_started":1}
I20260807 15:06:41.300146 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.295612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.18.129.3:33251)
0807 15:06:41.295874 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.295879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.295981 (+ 102us) server_negotiation.cc:407] Connection header received
0807 15:06:41.296063 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.296065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.296083 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.296156 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:41.296369 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.297473 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.298718 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.299481 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.299546 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.299596 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.300011 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.300052 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":85,"threads_started":1}
I20260807 15:06:41.300535 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.295531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:45933 (local address 127.0.0.1:34452)
0807 15:06:41.296075 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:41.296083 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:41.296131 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:41.296289 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:41.296289 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:41.296291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:41.296327 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.296328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.297357 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.297358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.298657 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.298659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.299849 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.299850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.299905 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.299908 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:41.300422 (+ 514us) client_negotiation.cc:240] Negotiation successful
0807 15:06:41.300469 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":80,"threads_started":1}
I20260807 15:06:41.300470 20103 tablet_service.cc:1467] Processing CreateTablet for tablet 04c31de30e3347cba49bac82ec97c783 (DEFAULT_TABLE table=test-table [id=203458ae861f43e6a525e837eaf0c678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:41.300827 20103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c31de30e3347cba49bac82ec97c783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:41.300766 20304 tablet_service.cc:1467] Processing CreateTablet for tablet 04c31de30e3347cba49bac82ec97c783 (DEFAULT_TABLE table=test-table [id=203458ae861f43e6a525e837eaf0c678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:41.301103 20304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c31de30e3347cba49bac82ec97c783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:41.303712 20419 tablet_bootstrap.cc:492] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14: Bootstrap starting.
I20260807 15:06:41.303629 20418 tablet_bootstrap.cc:492] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81: Bootstrap starting.
I20260807 15:06:41.304342 20419 tablet_bootstrap.cc:654] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:41.304243 19909 tablet_service.cc:1467] Processing CreateTablet for tablet 04c31de30e3347cba49bac82ec97c783 (DEFAULT_TABLE table=test-table [id=203458ae861f43e6a525e837eaf0c678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:41.304555 20418 tablet_bootstrap.cc:654] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:41.304577 19909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c31de30e3347cba49bac82ec97c783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:41.304984 20419 log.cc:824] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:41.305137 20418 log.cc:824] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:41.305677 20419 tablet_bootstrap.cc:492] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14: No bootstrap required, opened a new log
I20260807 15:06:41.305739 20419 ts_tablet_manager.cc:1397] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:41.307140 20419 raft_consensus.cc:348] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.307334 20419 raft_consensus.cc:374] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:41.307384 20419 raft_consensus.cc:729] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea21254339946f199218d124d20eb14, State: Initialized, Role: FOLLOWER
I20260807 15:06:41.307492 20419 consensus_queue.cc:260] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.307647 20418 tablet_bootstrap.cc:492] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81: No bootstrap required, opened a new log
I20260807 15:06:41.307629 20424 tablet_bootstrap.cc:492] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4: Bootstrap starting.
I20260807 15:06:41.307717 20418 ts_tablet_manager.cc:1397] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:06:41.307783 20419 ts_tablet_manager.cc:1428] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:41.307962 20391 heartbeater.cc:503] Master 127.18.129.62:34557 was elected leader, sending a full tablet report...
I20260807 15:06:41.308312 20424 tablet_bootstrap.cc:654] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:41.308908 20424 log.cc:824] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:41.309667 20424 tablet_bootstrap.cc:492] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4: No bootstrap required, opened a new log
I20260807 15:06:41.309741 20424 ts_tablet_manager.cc:1397] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:41.309755 20418 raft_consensus.cc:348] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.309939 20418 raft_consensus.cc:374] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:41.309988 20418 raft_consensus.cc:729] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0e532b7d2e424f81d29cc734a99c81, State: Initialized, Role: FOLLOWER
I20260807 15:06:41.310112 20418 consensus_queue.cc:260] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.310390 20418 ts_tablet_manager.cc:1428] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:06:41.310406 20202 heartbeater.cc:503] Master 127.18.129.62:34557 was elected leader, sending a full tablet report...
I20260807 15:06:41.311695 20424 raft_consensus.cc:348] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.311826 20424 raft_consensus.cc:374] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:41.311847 20424 raft_consensus.cc:729] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f0e32835954741825b21534b50ebe4, State: Initialized, Role: FOLLOWER
I20260807 15:06:41.311930 20424 consensus_queue.cc:260] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.312144 20424 ts_tablet_manager.cc:1428] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:06:41.312186 20008 heartbeater.cc:503] Master 127.18.129.62:34557 was elected leader, sending a full tablet report...
W20260807 15:06:41.351464 20203 tablet.cc:2369] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:41.382449 20430 raft_consensus.cc:482] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:41.382586 20430 raft_consensus.cc:504] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.382982 20430 leader_election.cc:288] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ea21254339946f199218d124d20eb14 (127.18.129.3:33251), aa0e532b7d2e424f81d29cc734a99c81 (127.18.129.2:37563)
I20260807 15:06:41.386775 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.383166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:37563 (local address 127.18.129.1:35959)
0807 15:06:41.383237 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:41.383252 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:41.383300 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:41.383494 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:41.383495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:41.383497 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:41.383579 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.383581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.384476 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.384477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.385766 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.385769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.386511 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.386513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.386604 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.386608 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:41.386651 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:06:41.386695 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":57}
I20260807 15:06:41.386783 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.383256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:35959 (local address 127.18.129.2:37563)
0807 15:06:41.383311 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.383313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.383323 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:41.383355 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.383356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.383367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.383481 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:41.383645 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.384396 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.385856 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.386438 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.386497 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.386507 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.386669 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.386710 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:06:41.386835 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.383212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:42903 (local address 127.18.129.3:33251)
0807 15:06:41.383277 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.383279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.383495 (+ 216us) server_negotiation.cc:407] Connection header received
0807 15:06:41.383530 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.383531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.383544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.383594 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:41.383718 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.384400 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.385856 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.386426 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.386500 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.386507 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.386737 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.386778 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:06:41.387106 20432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.383167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:33251 (local address 127.18.129.1:42903)
0807 15:06:41.383424 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:41.383432 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:41.383485 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:41.383602 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:41.383603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:41.383604 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:41.383662 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.383663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.384476 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.384477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.385766 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:41.385769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.386564 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.386566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:41.386655 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.386660 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:06:41.386983 (+ 323us) client_negotiation.cc:240] Negotiation successful
0807 15:06:41.387026 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":122,"thread_start_us":121,"threads_started":1}
I20260807 15:06:41.387405 20133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c31de30e3347cba49bac82ec97c783" candidate_uuid: "56f0e32835954741825b21534b50ebe4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0e532b7d2e424f81d29cc734a99c81" is_pre_election: true
I20260807 15:06:41.387507 20330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c31de30e3347cba49bac82ec97c783" candidate_uuid: "56f0e32835954741825b21534b50ebe4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea21254339946f199218d124d20eb14" is_pre_election: true
I20260807 15:06:41.387562 20133 raft_consensus.cc:2454] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56f0e32835954741825b21534b50ebe4 in term 0.
I20260807 15:06:41.387652 20330 raft_consensus.cc:2454] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56f0e32835954741825b21534b50ebe4 in term 0.
I20260807 15:06:41.387795 19851 leader_election.cc:302] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f0e32835954741825b21534b50ebe4, aa0e532b7d2e424f81d29cc734a99c81; no voters:
I20260807 15:06:41.387944 20430 raft_consensus.cc:2781] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:41.388003 20430 raft_consensus.cc:482] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:41.388018 20430 raft_consensus.cc:3037] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:41.388659 20430 raft_consensus.cc:504] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.388769 20430 leader_election.cc:288] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [CANDIDATE]: Term 1 election: Requested vote from peers 8ea21254339946f199218d124d20eb14 (127.18.129.3:33251), aa0e532b7d2e424f81d29cc734a99c81 (127.18.129.2:37563)
I20260807 15:06:41.388929 20330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c31de30e3347cba49bac82ec97c783" candidate_uuid: "56f0e32835954741825b21534b50ebe4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea21254339946f199218d124d20eb14"
I20260807 15:06:41.388931 20133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c31de30e3347cba49bac82ec97c783" candidate_uuid: "56f0e32835954741825b21534b50ebe4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0e532b7d2e424f81d29cc734a99c81"
I20260807 15:06:41.389012 20330 raft_consensus.cc:3037] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:41.389014 20133 raft_consensus.cc:3037] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:41.389871 20330 raft_consensus.cc:2454] T 04c31de30e3347cba49bac82ec97c783 P 8ea21254339946f199218d124d20eb14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56f0e32835954741825b21534b50ebe4 in term 1.
I20260807 15:06:41.389871 20133 raft_consensus.cc:2454] T 04c31de30e3347cba49bac82ec97c783 P aa0e532b7d2e424f81d29cc734a99c81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56f0e32835954741825b21534b50ebe4 in term 1.
I20260807 15:06:41.390035 19850 leader_election.cc:302] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f0e32835954741825b21534b50ebe4, 8ea21254339946f199218d124d20eb14; no voters:
I20260807 15:06:41.390159 20430 raft_consensus.cc:2781] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:41.390318 20430 raft_consensus.cc:686] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [term 1 LEADER]: Becoming Leader. State: Replica: 56f0e32835954741825b21534b50ebe4, State: Running, Role: LEADER
I20260807 15:06:41.390398 20430 consensus_queue.cc:237] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } }
I20260807 15:06:41.391319 19743 catalog_manager.cc:5495] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56f0e32835954741825b21534b50ebe4 (127.18.129.1). New cstate: current_term: 1 leader_uuid: "56f0e32835954741825b21534b50ebe4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea21254339946f199218d124d20eb14" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56f0e32835954741825b21534b50ebe4" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa0e532b7d2e424f81d29cc734a99c81" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37563 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:06:41.417882 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.410961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34464 (local address 127.18.129.1:45933)
0807 15:06:41.411014 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.411016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.411142 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:06:41.411206 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.411208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.411221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.411269 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:41.411798 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.412263 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.412944 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.413134 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.413189 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.415518 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:41.415530 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:41.415530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:41.415537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:41.416946 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.417251 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.417253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.417254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.417313 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.417459 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.417459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.417460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.417597 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:41.417714 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.417742 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.417807 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115201, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260807 15:06:41.433681 20011 tablet.cc:2369] T 04c31de30e3347cba49bac82ec97c783 P 56f0e32835954741825b21534b50ebe4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115201, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:06:41.440626 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.434289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.18.129.1:45933)
0807 15:06:41.434339 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.434341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.434437 (+ 96us) server_negotiation.cc:407] Connection header received
0807 15:06:41.434505 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.434505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.434516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.434570 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:41.435149 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.435681 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.436380 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.436586 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.436641 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.438649 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:41.438651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:41.438651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:41.438657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:41.439771 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.440047 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.440048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.440048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.440093 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.440239 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.440240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.440240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.440320 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:41.440397 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.440491 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.440559 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[19682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:06:41.461967 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:41.456083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.18.129.1:45933)
0807 15:06:41.456144 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:41.456146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:41.456231 (+ 85us) server_negotiation.cc:407] Connection header received
0807 15:06:41.456279 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:41.456280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:41.456289 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:41.456334 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:41.456816 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.457271 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.457926 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:41.458103 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:41.458146 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:41.460173 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:41.460174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:41.460175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:41.460181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:41.461202 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.461468 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.461468 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.461469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.461510 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:41.461634 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:41.461635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:41.461635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:41.461711 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:41.461777 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:41.461841 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 15:06:41.461894 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 15:06:41.462730 19885 tablet_service.cc:2908] Rejecting scan request for tablet 04c31de30e3347cba49bac82ec97c783: Uninitialized: clean time has not yet been initialized
I20260807 15:06:41.463646 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19826
I20260807 15:06:41.477262 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20015
I20260807 15:06:41.487619 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20207
I20260807 15:06:41.498059 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19700
2026-08-07T15:06:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2119 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[20452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[20452](info): set up 2 sockets
Aug 07 15:06:41 dist-test-slave-5bn2 krb5kdc[20452](info): commencing operation
krb5kdc: starting...
W20260807 15:06:42.630748 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:42 dist-test-slave-5bn2 krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:42.753964 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:40825
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:40825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:42.838380 20475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:42.838604 20475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:42.838636 20475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:42.839823 20475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:42.839869 20475 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:06:42.839885 20475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:42.839902 20475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:42.839918 20475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:42.839941 20475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:42.841439 20475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:40825
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:42.841691 20475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:42.841895 20475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:42.847000 20483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:42.847019 20480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:42.847152 20475 server_base.cc:1034] running on GCE node
W20260807 15:06:42.847205 20481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:42.847519 20475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:42.847795 20475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:42.848939 20475 hybrid_clock.cc:648] HybridClock initialized: now 1786115202848918 us; error 33 us; skew 500 ppm
Aug 07 15:06:42 dist-test-slave-5bn2 krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115202, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:42.855943 20475 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:42.856411 20475 webserver.cc:460] Webserver started at http://127.18.129.62:38625/ using document root <none> and password file <none>
I20260807 15:06:42.856653 20475 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:42.856706 20475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:42.856835 20475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:42.857910 20475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa"
format_stamp: "Formatted at 2026-08-07 15:06:42 on dist-test-slave-5bn2"
server_key: "fa9b77e3e0b5f978236048a8aeaf737f"
server_key_iv: "f8ceb9aeebcf4ec021f6a7ed4968797d"
server_key_version: "encryptionkey@0"
I20260807 15:06:42.858276 20475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa"
format_stamp: "Formatted at 2026-08-07 15:06:42 on dist-test-slave-5bn2"
server_key: "fa9b77e3e0b5f978236048a8aeaf737f"
server_key_iv: "f8ceb9aeebcf4ec021f6a7ed4968797d"
server_key_version: "encryptionkey@0"
I20260807 15:06:42.859604 20475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 15:06:42.860289 20493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:42.860437 20475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:42.860514 20475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa"
format_stamp: "Formatted at 2026-08-07 15:06:42 on dist-test-slave-5bn2"
server_key: "fa9b77e3e0b5f978236048a8aeaf737f"
server_key_iv: "f8ceb9aeebcf4ec021f6a7ed4968797d"
server_key_version: "encryptionkey@0"
I20260807 15:06:42.860577 20475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:42.875139 20475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:42.876869 20475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:42.877012 20475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:42.881712 20475 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:40825
I20260807 15:06:42.881775 20575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:40825 every 8 connection(s)
I20260807 15:06:42.882089 20475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:42.882525 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20475
I20260807 15:06:42.882602 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:42.882762 18948 external_mini_cluster.cc:1428] Setting key d0b15dc9ca9fd352094a628284855955
I20260807 15:06:42.882755 20578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:42.886148 20578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa: Bootstrap starting.
I20260807 15:06:42.886796 20578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:42.887174 20578 log.cc:824] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:42.887892 20578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa: No bootstrap required, opened a new log
Aug 07 15:06:42 dist-test-slave-5bn2 krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:42.889973 20578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40825 } }
I20260807 15:06:42.890174 20578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:42.890257 20578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b59e5da93f4cd7bc03c64d0e12c9aa, State: Initialized, Role: FOLLOWER
I20260807 15:06:42.890403 20578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40825 } }
I20260807 15:06:42.890479 20578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:42.890503 20578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:42.890550 20578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:42.891184 20578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40825 } }
I20260807 15:06:42.891276 20578 leader_election.cc:302] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44b59e5da93f4cd7bc03c64d0e12c9aa; no voters:
I20260807 15:06:42.891440 20578 leader_election.cc:288] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:42.891492 20590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:42.891690 20578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:42.891705 20590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [term 1 LEADER]: Becoming Leader. State: Replica: 44b59e5da93f4cd7bc03c64d0e12c9aa, State: Running, Role: LEADER
I20260807 15:06:42.891902 20590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40825 } }
I20260807 15:06:42.892295 20594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44b59e5da93f4cd7bc03c64d0e12c9aa. Latest consensus state: current_term: 1 leader_uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40825 } } }
I20260807 15:06:42.892385 20594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:42.892534 20592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b59e5da93f4cd7bc03c64d0e12c9aa" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40825 } } }
I20260807 15:06:42.892577 20604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:42.892594 20592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:42.893090 20604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:42.893491 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:42.883307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46814 (local address 127.18.129.62:40825)
0807 15:06:42.883505 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:42.883516 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:42.883559 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:06:42.883974 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:42.883984 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:42.884049 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:42.884203 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:42.884773 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:42.886107 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:42.886871 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:42.887462 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:42.887577 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:42.889804 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:42.889808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:42.889809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:42.889819 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:42.891558 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:42.892026 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:42.892029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:42.892029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:42.892138 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:42.892359 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:42.892359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:42.892360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:42.892693 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:42.892860 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:42.893142 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 15:06:42.893280 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260807 15:06:42.894711 20604 catalog_manager.cc:1347] Generated new cluster ID: 16002840e99449b394a4f82b70a7a0e2
I20260807 15:06:42.894764 20604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:42.942528 20604 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:42.943141 20604 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:42.945900 20604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa: Generated new TSK 0
I20260807 15:06:42.946070 20604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:06:42.978809 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40825
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:43.074498 20624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:43.074697 20624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:43.074719 20624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:43.075883 20624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:43.075929 20624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:43.075950 20624 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:06:43.075982 20624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:43.077421 20624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:40825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:43.077641 20624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:43.077857 20624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:43.082813 20633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:43.082827 20632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:43.082999 20624 server_base.cc:1034] running on GCE node
W20260807 15:06:43.082826 20637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:43.083269 20624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:43.083508 20624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:43.084647 20624 hybrid_clock.cc:648] HybridClock initialized: now 1786115203084631 us; error 34 us; skew 500 ppm
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:43.087080 20624 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:06:43.087486 20624 webserver.cc:460] Webserver started at http://127.18.129.1:45861/ using document root <none> and password file <none>
I20260807 15:06:43.087720 20624 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:43.087769 20624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:43.087893 20624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:43.088901 20624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "33bacb23b6e34f859b72dd547eeb04c9"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "d082e0f243d7cc7007995f34f77b6c0b"
server_key_iv: "100ce937bf7f4a20606f8fb6cf203b96"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.089231 20624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "33bacb23b6e34f859b72dd547eeb04c9"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "d082e0f243d7cc7007995f34f77b6c0b"
server_key_iv: "100ce937bf7f4a20606f8fb6cf203b96"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.090440 20624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:43.091146 20648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.091291 20624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.091377 20624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "33bacb23b6e34f859b72dd547eeb04c9"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "d082e0f243d7cc7007995f34f77b6c0b"
server_key_iv: "100ce937bf7f4a20606f8fb6cf203b96"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.091436 20624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:43.105646 20624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:43.107344 20624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:43.107479 20624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:43.107975 20624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:43.108013 20624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.108043 20624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:43.108062 20624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.114401 20624 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:43895
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:43.114465 20808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:43895 every 8 connection(s)
I20260807 15:06:43.114882 20624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:06:43.115069 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20624
I20260807 15:06:43.115172 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:43.115656 18948 external_mini_cluster.cc:1428] Setting key faa8cad869fde65a2db3751edd514621
I20260807 15:06:43.117774 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.108570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:37809 (local address 127.18.129.62:40825)
0807 15:06:43.108639 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.108641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.108779 (+ 138us) server_negotiation.cc:407] Connection header received
0807 15:06:43.109266 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.109267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.109283 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.109357 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.110188 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.110967 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.111980 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.112302 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.112381 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.115032 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.115033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.115033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.115041 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.116488 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.116916 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.116917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.116917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.116966 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.117207 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.117208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.117208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.117302 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.117348 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.117633 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.117707 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:06:43.117995 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.108488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40825 (local address 127.18.129.1:37809)
0807 15:06:43.108655 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.108696 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.109154 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.109432 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.109437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.109791 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:43.110068 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.110074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.111065 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.111066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.111910 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.111913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.112401 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.112402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.112550 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.113070 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:43.113079 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:43.114922 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:43.116567 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.116576 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.116578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.116847 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.117013 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.117013 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.117014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.117151 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.117351 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:43.117357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:43.117506 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.117620 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.117780 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260807 15:06:43.118489 20810 heartbeater.cc:347] Connected to a master server at 127.18.129.62:40825
I20260807 15:06:43.118583 20810 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:43.118788 20810 heartbeater.cc:511] Master 127.18.129.62:40825 requested a full tablet report, sending...
I20260807 15:06:43.119380 20515 ts_manager.cc:194] Registered new tserver with Master: 33bacb23b6e34f859b72dd547eeb04c9 (127.18.129.1:43895)
I20260807 15:06:43.119989 20515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:37809
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
I20260807 15:06:43.125771 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.120404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:56781 (local address 127.18.129.62:40825)
0807 15:06:43.120453 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.120455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.120464 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:06:43.120525 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.120525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.120535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.120612 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.121018 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.121480 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.122432 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.122619 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.122663 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.123613 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.123615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.123615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.123621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.124819 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.125181 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.125182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.125182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.125235 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.125409 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.125410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.125410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.125498 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.125542 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.125652 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.125712 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:06:43.125780 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.120285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40825 (local address 127.18.129.1:56781)
0807 15:06:43.120416 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.120426 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.120476 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.120656 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.120658 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.120863 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:43.120949 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.120951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.121589 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.121590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.122355 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.122358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.122681 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.122682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.122756 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.123143 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:43.123147 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:43.123529 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:43.124896 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.124897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.124897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.125106 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.125289 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.125290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.125290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.125355 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.125532 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:43.125533 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:43.125591 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.125635 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.125709 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:06:43.142508 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40825
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:43.225971 20826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:43.226197 20826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:43.226227 20826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:43.227450 20826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:43.227494 20826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:43.227521 20826 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:06:43.227545 20826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:43.228991 20826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:40825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:43.229231 20826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:43.229453 20826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:43.234591 20837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:43.234618 20836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:43.234690 20826 server_base.cc:1034] running on GCE node
W20260807 15:06:43.234586 20840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:43.235023 20826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:43.235249 20826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:43.236392 20826 hybrid_clock.cc:648] HybridClock initialized: now 1786115203236351 us; error 55 us; skew 500 ppm
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:43.238843 20826 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:06:43.239289 20826 webserver.cc:460] Webserver started at http://127.18.129.2:40819/ using document root <none> and password file <none>
I20260807 15:06:43.239470 20826 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:43.239511 20826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:43.239599 20826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:43.240600 20826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "3a78167f087944a49f68d584decc98d7"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "6d8f42565058e3578a58ac6e626bd559"
server_key_iv: "efbadc0c9a7c10a7ca9ac0310ce0679b"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.240911 20826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "3a78167f087944a49f68d584decc98d7"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "6d8f42565058e3578a58ac6e626bd559"
server_key_iv: "efbadc0c9a7c10a7ca9ac0310ce0679b"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.242133 20826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:06:43.242839 20847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.243053 20826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:43.243150 20826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "3a78167f087944a49f68d584decc98d7"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "6d8f42565058e3578a58ac6e626bd559"
server_key_iv: "efbadc0c9a7c10a7ca9ac0310ce0679b"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.243219 20826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:43.266971 20826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:43.268865 20826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:43.269011 20826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:43.269541 20826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:43.269583 20826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.269613 20826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:43.269631 20826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.276337 20826 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:40475
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:43.276762 20826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:06:43.276886 21032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:40475 every 8 connection(s)
I20260807 15:06:43.277670 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20826
I20260807 15:06:43.277761 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:43.277946 18948 external_mini_cluster.cc:1428] Setting key 47a5687c7a72c97da07286444841ff73
I20260807 15:06:43.281036 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.269930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40825 (local address 127.18.129.2:57241)
0807 15:06:43.270078 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.270131 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.270635 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.270940 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.270945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.271237 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:43.271430 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.271435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.272373 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.272374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.273637 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.273640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.274141 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.274142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.274316 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.274833 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:43.274843 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:43.277340 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:43.279181 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.279187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.279189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.279472 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.279728 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.279729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.279730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.279844 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.280125 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:43.280160 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:43.280410 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.280591 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.280797 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260807 15:06:43.281335 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.269981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:57241 (local address 127.18.129.62:40825)
0807 15:06:43.270049 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.270052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.270221 (+ 169us) server_negotiation.cc:407] Connection header received
0807 15:06:43.270729 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.270731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.270744 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.270818 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.271509 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.272258 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.273739 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.274042 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.274122 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.277441 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.277443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.277443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.277452 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.279100 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.279559 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.279560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.279560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.279637 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.279915 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.279916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.279916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.280060 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.280136 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.281166 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.281248 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:06:43.281690 21034 heartbeater.cc:347] Connected to a master server at 127.18.129.62:40825
I20260807 15:06:43.281774 21034 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:43.281968 21034 heartbeater.cc:511] Master 127.18.129.62:40825 requested a full tablet report, sending...
I20260807 15:06:43.282367 20516 ts_manager.cc:194] Registered new tserver with Master: 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475)
I20260807 15:06:43.282943 20516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:57241
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
I20260807 15:06:43.288411 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.283065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:51225 (local address 127.18.129.62:40825)
0807 15:06:43.283155 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.283156 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.283164 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:43.283181 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.283181 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.283189 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.283228 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.283691 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.284248 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.285038 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.285234 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.285287 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.286127 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.286128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.286129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.286135 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.287403 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.287698 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.287699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.287700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.287751 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.287910 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.287911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.287911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.288059 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.288123 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.288292 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.288352 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:06:43.288475 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.282725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40825 (local address 127.18.129.2:51225)
0807 15:06:43.282913 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.282924 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.282973 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.283269 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.283270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.283527 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:43.283608 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.283610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.284354 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.284355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.284962 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.284965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.285292 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.285293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.285368 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.285766 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:43.285771 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:43.286046 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:43.287477 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.287478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.287478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.287640 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.287803 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.287803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.287804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.287861 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.288126 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:43.288127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:43.288184 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.288277 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.288385 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:06:43.305918 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40825
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:43.387683 21049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:43.387882 21049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:43.387907 21049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:43.388998 21049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:43.389031 21049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:43.389050 21049 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:06:43.389066 21049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:43.390488 21049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:40825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:43.390666 21049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:43.390831 21049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:43.395900 21056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:43.395943 21058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:43.395929 21054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:43.396126 21049 server_base.cc:1034] running on GCE node
I20260807 15:06:43.396317 21049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:43.396559 21049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:43.397715 21049 hybrid_clock.cc:648] HybridClock initialized: now 1786115203397697 us; error 29 us; skew 500 ppm
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:43.400255 21049 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:06:43.400734 21049 webserver.cc:460] Webserver started at http://127.18.129.3:35273/ using document root <none> and password file <none>
I20260807 15:06:43.400933 21049 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:43.400988 21049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:43.401113 21049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:43.402146 21049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "52274734f1ac41109986701bfacfa3fc"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "877b9fda96a53a54d9b8f5a4585c5820"
server_key_iv: "dc066b7c7f7f6739999d82a99d2bc5c4"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.402485 21049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "52274734f1ac41109986701bfacfa3fc"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "877b9fda96a53a54d9b8f5a4585c5820"
server_key_iv: "dc066b7c7f7f6739999d82a99d2bc5c4"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.403784 21049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:43.404567 21073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.404721 21049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:06:43.404807 21049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "52274734f1ac41109986701bfacfa3fc"
format_stamp: "Formatted at 2026-08-07 15:06:43 on dist-test-slave-5bn2"
server_key: "877b9fda96a53a54d9b8f5a4585c5820"
server_key_iv: "dc066b7c7f7f6739999d82a99d2bc5c4"
server_key_version: "encryptionkey@0"
I20260807 15:06:43.404865 21049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:43.417958 21049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:43.419819 21049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:43.419956 21049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:43.420444 21049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:43.420485 21049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.420513 21049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:43.420531 21049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:43.427619 21049 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:33127
I20260807 15:06:43.427702 21232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:33127 every 8 connection(s)
I20260807 15:06:43.428028 21049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:06:43.430234 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.420880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:51637 (local address 127.18.129.62:40825)
0807 15:06:43.420946 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.420949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.421162 (+ 213us) server_negotiation.cc:407] Connection header received
0807 15:06:43.421831 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.421833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.421846 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.421909 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.422732 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.423323 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.424786 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.425003 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.425054 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.427539 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.427541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.427541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.427548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.428987 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.429347 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.429348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.429348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.429408 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.429622 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.429623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.429624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.429737 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.429793 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.430039 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.430142 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:06:43.430371 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.420847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40825 (local address 127.18.129.3:51637)
0807 15:06:43.421020 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.421071 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.421731 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.421990 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.421994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.422344 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:43.422646 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.422652 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.423401 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.423402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.424700 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.424704 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.425086 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.425087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.425231 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.425772 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:43.425783 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:43.427448 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:43.429060 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.429067 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.429069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.429280 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.429459 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.429460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.429460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.429558 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.429791 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:43.429798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:43.429924 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.430009 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.430172 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260807 15:06:43.430796 21233 heartbeater.cc:347] Connected to a master server at 127.18.129.62:40825
I20260807 15:06:43.430979 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 21049
I20260807 15:06:43.431067 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:43.431098 21233 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:43.431241 18948 external_mini_cluster.cc:1428] Setting key ad51b5f0bc8f107ef392df8e7276720a
I20260807 15:06:43.431288 21233 heartbeater.cc:511] Master 127.18.129.62:40825 requested a full tablet report, sending...
I20260807 15:06:43.431618 20515 ts_manager.cc:194] Registered new tserver with Master: 52274734f1ac41109986701bfacfa3fc (127.18.129.3:33127)
I20260807 15:06:43.432087 20515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:51637
I20260807 15:06:43.432108 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:06:43.436902 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.431910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:36747 (local address 127.18.129.62:40825)
0807 15:06:43.431978 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.431980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.431986 (+ 6us) server_negotiation.cc:407] Connection header received
0807 15:06:43.432005 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.432006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.432014 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.432059 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.432387 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.432841 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.433558 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.433755 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.433807 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.434685 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.434687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.434687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.434695 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.435930 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.436233 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.436234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.436234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.436290 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.436448 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.436449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.436449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.436552 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.436609 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.436751 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.436814 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:06:43.436916 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.431763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40825 (local address 127.18.129.3:36747)
0807 15:06:43.431881 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.431892 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.431940 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.432061 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.432062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.432252 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:43.432318 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.432320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.432985 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.432985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.433486 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.433488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.433813 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.433814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.433887 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.434257 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:43.434262 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:43.434601 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:43.436005 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.436006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.436007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.436172 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.436341 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.436342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.436342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.436399 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.436607 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:43.436608 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:43.436648 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.436719 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.436813 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:43.455478 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.449271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.18.129.62:40825)
0807 15:06:43.449314 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.449315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.449323 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:43.449374 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.449374 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.449382 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.449427 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.449931 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.450434 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.451187 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.451369 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.451419 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.453394 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.453395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.453395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.453402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.454463 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.454777 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.454778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.454778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.454827 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.455006 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.455007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.455008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.455116 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.455200 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.455317 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.455390 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:06:43.456914 20515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:43.458410 20515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:43.464637 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.460497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:43895 (local address 127.0.0.1:39106)
0807 15:06:43.460685 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.460727 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.460790 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.461191 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.461192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.461195 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:43.461248 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.461250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.462312 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.462313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.463546 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.463549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.464344 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.464345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.464425 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.464438 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.464506 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.464561 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":43,"threads_started":1}
I20260807 15:06:43.464779 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.460799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.18.129.1:43895)
0807 15:06:43.460979 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.460983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.460992 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:06:43.461024 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.461026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.461054 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.461124 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:43.461330 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.462229 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.463656 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.464269 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.464331 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.464392 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.464558 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.464600 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260807 15:06:43.464835 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.460739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.18.129.3:33127)
0807 15:06:43.460886 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.460890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.460934 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:06:43.461019 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.461021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.461036 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.461118 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:43.461405 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.462517 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.463631 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.464293 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.464361 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.464412 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.464695 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.464737 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260807 15:06:43.465070 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.460497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:33127 (local address 127.0.0.1:52410)
0807 15:06:43.460886 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.460894 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.460938 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.461138 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.461141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.461143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:43.461209 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.461211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.462610 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.462611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.463559 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.463562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.464450 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.464451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.464540 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.464545 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.464605 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.464646 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":118,"threads_started":1}
I20260807 15:06:43.465262 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.460499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:40475 (local address 127.0.0.1:46224)
0807 15:06:43.461258 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.461267 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.461318 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.461463 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.461464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.461466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:43.461515 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.461517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.462729 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.462729 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.463857 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.463860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.464797 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.464798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.464856 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.464860 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.465148 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.465196 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":120,"threads_started":1}
I20260807 15:06:43.465549 20712 tablet_service.cc:1467] Processing CreateTablet for tablet 8480293812414b769cbf5bc3c20262aa (DEFAULT_TABLE table=test-table [id=6253400bdc0242299682a92f04e21215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:43.465812 21141 tablet_service.cc:1467] Processing CreateTablet for tablet 8480293812414b769cbf5bc3c20262aa (DEFAULT_TABLE table=test-table [id=6253400bdc0242299682a92f04e21215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:43.465940 20712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8480293812414b769cbf5bc3c20262aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.466148 21141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8480293812414b769cbf5bc3c20262aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.467080 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.461008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46224 (local address 127.18.129.2:40475)
0807 15:06:43.461215 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.461220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.461323 (+ 103us) server_negotiation.cc:407] Connection header received
0807 15:06:43.461375 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.461377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.461392 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.461468 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:43.461604 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.462654 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.463937 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.464651 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.464710 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.464761 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.466932 (+ 2171us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.466980 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
I20260807 15:06:43.469019 21279 tablet_bootstrap.cc:492] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Bootstrap starting.
I20260807 15:06:43.468971 21277 tablet_bootstrap.cc:492] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: Bootstrap starting.
I20260807 15:06:43.469282 20933 tablet_service.cc:1467] Processing CreateTablet for tablet 8480293812414b769cbf5bc3c20262aa (DEFAULT_TABLE table=test-table [id=6253400bdc0242299682a92f04e21215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:43.469534 20933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8480293812414b769cbf5bc3c20262aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.469729 21279 tablet_bootstrap.cc:654] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.469969 21277 tablet_bootstrap.cc:654] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.470036 21279 log.cc:824] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:43.470429 21277 log.cc:824] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:43.470937 21279 tablet_bootstrap.cc:492] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: No bootstrap required, opened a new log
I20260807 15:06:43.470999 21279 ts_tablet_manager.cc:1397] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:43.471448 21277 tablet_bootstrap.cc:492] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: No bootstrap required, opened a new log
I20260807 15:06:43.471506 21277 ts_tablet_manager.cc:1397] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:06:43.472688 21285 tablet_bootstrap.cc:492] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Bootstrap starting.
I20260807 15:06:43.473242 21285 tablet_bootstrap.cc:654] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.473233 21279 raft_consensus.cc:348] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.473425 21279 raft_consensus.cc:374] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.473353 21277 raft_consensus.cc:348] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.473470 21279 raft_consensus.cc:729] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33bacb23b6e34f859b72dd547eeb04c9, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.473490 21277 raft_consensus.cc:374] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.473529 21277 raft_consensus.cc:729] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52274734f1ac41109986701bfacfa3fc, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.473555 21285 log.cc:824] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:43.473626 21277 consensus_queue.cc:260] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.473632 21279 consensus_queue.cc:260] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.473901 21277 ts_tablet_manager.cc:1428] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:06:43.473901 21279 ts_tablet_manager.cc:1428] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:06:43.473935 20810 heartbeater.cc:503] Master 127.18.129.62:40825 was elected leader, sending a full tablet report...
I20260807 15:06:43.474057 21233 heartbeater.cc:503] Master 127.18.129.62:40825 was elected leader, sending a full tablet report...
I20260807 15:06:43.474545 21285 tablet_bootstrap.cc:492] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: No bootstrap required, opened a new log
I20260807 15:06:43.474613 21285 ts_tablet_manager.cc:1397] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:06:43.476061 21285 raft_consensus.cc:348] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.476204 21285 raft_consensus.cc:374] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.476248 21285 raft_consensus.cc:729] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a78167f087944a49f68d584decc98d7, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.476357 21285 consensus_queue.cc:260] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.476630 21285 ts_tablet_manager.cc:1428] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:06:43.476655 21034 heartbeater.cc:503] Master 127.18.129.62:40825 was elected leader, sending a full tablet report...
W20260807 15:06:43.527441 21036 tablet.cc:2369] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:06:43.615758 20811 tablet.cc:2369] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:43.616230 21289 raft_consensus.cc:482] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:43.616345 21289 raft_consensus.cc:504] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.616711 21289 leader_election.cc:288] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33bacb23b6e34f859b72dd547eeb04c9 (127.18.129.1:43895), 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475)
I20260807 15:06:43.619524 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.616941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:36855 (local address 127.18.129.1:43895)
0807 15:06:43.616983 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.616985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.616995 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:43.617093 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.617094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.617104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.617149 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:43.617296 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.617762 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.618747 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.619199 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.619275 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.619284 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.619421 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.619454 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:06:43.619524 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.616902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:43895 (local address 127.18.129.3:36855)
0807 15:06:43.616949 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.616961 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.617006 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.617159 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.617159 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.617161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:43.617241 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.617242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.617833 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.617834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.618664 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.618667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.619281 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.619282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.619361 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.619365 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.619410 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.619448 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 15:06:43.619681 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.616973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:34125 (local address 127.18.129.2:40475)
0807 15:06:43.617021 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.617023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.617225 (+ 202us) server_negotiation.cc:407] Connection header received
0807 15:06:43.617284 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.617285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.617294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.617342 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:43.617471 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.617969 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.618947 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.619340 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.619402 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.619410 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.619536 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.619570 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:06:43.619837 20744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8480293812414b769cbf5bc3c20262aa" candidate_uuid: "52274734f1ac41109986701bfacfa3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33bacb23b6e34f859b72dd547eeb04c9" is_pre_election: true
I20260807 15:06:43.619978 20744 raft_consensus.cc:2454] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52274734f1ac41109986701bfacfa3fc in term 0.
I20260807 15:06:43.619988 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.616903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:40475 (local address 127.18.129.3:34125)
0807 15:06:43.617155 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.617163 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.617232 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.617349 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.617350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.617351 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:43.617416 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.617418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.618037 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.618038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.618870 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.618873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.619406 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.619407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.619478 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.619481 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.619525 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.619563 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":52,"thread_start_us":136,"threads_started":1}
I20260807 15:06:43.620210 21076 leader_election.cc:302] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33bacb23b6e34f859b72dd547eeb04c9, 52274734f1ac41109986701bfacfa3fc; no voters:
I20260807 15:06:43.620329 21289 raft_consensus.cc:2781] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:43.620383 21289 raft_consensus.cc:482] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:43.620402 21289 raft_consensus.cc:3037] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.620396 20964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8480293812414b769cbf5bc3c20262aa" candidate_uuid: "52274734f1ac41109986701bfacfa3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a78167f087944a49f68d584decc98d7" is_pre_election: true
I20260807 15:06:43.620546 20964 raft_consensus.cc:2454] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52274734f1ac41109986701bfacfa3fc in term 0.
I20260807 15:06:43.621521 21289 raft_consensus.cc:504] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.621657 21289 leader_election.cc:288] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [CANDIDATE]: Term 1 election: Requested vote from peers 33bacb23b6e34f859b72dd547eeb04c9 (127.18.129.1:43895), 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475)
I20260807 15:06:43.621788 20744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8480293812414b769cbf5bc3c20262aa" candidate_uuid: "52274734f1ac41109986701bfacfa3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33bacb23b6e34f859b72dd547eeb04c9"
I20260807 15:06:43.621814 20964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8480293812414b769cbf5bc3c20262aa" candidate_uuid: "52274734f1ac41109986701bfacfa3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a78167f087944a49f68d584decc98d7"
I20260807 15:06:43.621863 20744 raft_consensus.cc:3037] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.621896 20964 raft_consensus.cc:3037] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.622599 20964 raft_consensus.cc:2454] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52274734f1ac41109986701bfacfa3fc in term 1.
I20260807 15:06:43.622613 20744 raft_consensus.cc:2454] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52274734f1ac41109986701bfacfa3fc in term 1.
I20260807 15:06:43.622779 21075 leader_election.cc:302] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a78167f087944a49f68d584decc98d7, 52274734f1ac41109986701bfacfa3fc; no voters:
I20260807 15:06:43.622892 21289 raft_consensus.cc:2781] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:43.623023 21289 raft_consensus.cc:686] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 1 LEADER]: Becoming Leader. State: Replica: 52274734f1ac41109986701bfacfa3fc, State: Running, Role: LEADER
I20260807 15:06:43.623112 21289 consensus_queue.cc:237] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.623963 20515 catalog_manager.cc:5495] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 52274734f1ac41109986701bfacfa3fc (127.18.129.3). New cstate: current_term: 1 leader_uuid: "52274734f1ac41109986701bfacfa3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } health_report { overall_health: HEALTHY } } }
I20260807 15:06:43.645872 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.643739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.18.129.3:33127)
0807 15:06:43.643781 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.643782 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.643790 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:43.643886 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.643887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.643902 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.643956 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:43.644064 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.644538 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.645237 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.645409 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.645450 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.645593 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:43.645682 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:43.645730 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.645767 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.645787 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:06:43.647612 20744 raft_consensus.cc:1261] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Refusing update from remote peer 52274734f1ac41109986701bfacfa3fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:43.647622 20964 raft_consensus.cc:1261] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Refusing update from remote peer 52274734f1ac41109986701bfacfa3fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:43.647943 21289 consensus_queue.cc:1035] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:43.648080 21293 consensus_queue.cc:1035] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:43.650607 21299 mvcc.cc:205] Tried to move back new op lower bound from 7315927874138095616 to 7315927874040889344. Current Snapshot: MvccSnapshot[applied={T|T < 7315927874138095616}]
I20260807 15:06:43.650632 21298 mvcc.cc:205] Tried to move back new op lower bound from 7315927874138095616 to 7315927874040889344. Current Snapshot: MvccSnapshot[applied={T|T < 7315927874138095616}]
I20260807 15:06:43.650717 21300 mvcc.cc:205] Tried to move back new op lower bound from 7315927874138095616 to 7315927874040889344. Current Snapshot: MvccSnapshot[applied={T|T < 7315927874138095616}]
I20260807 15:06:43.654731 20515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:43.654912 20515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:43.655777 20515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa: Sending DeleteTablet for 3 replicas of tablet 8480293812414b769cbf5bc3c20262aa
I20260807 15:06:43.656252 21141 tablet_service.cc:1514] Processing DeleteTablet for tablet 8480293812414b769cbf5bc3c20262aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:43 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:52410
I20260807 15:06:43.656293 20515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:43.656404 20515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:43.657639 21141 tablet_service.cc:1467] Processing CreateTablet for tablet 19179f35b2424ac997deefeaa703bf51 (DEFAULT_TABLE table=test-table [id=45ecd47dc11b4bb296321310031f3578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:43.657694 20932 tablet_service.cc:1467] Processing CreateTablet for tablet 19179f35b2424ac997deefeaa703bf51 (DEFAULT_TABLE table=test-table [id=45ecd47dc11b4bb296321310031f3578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:43.657761 21141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19179f35b2424ac997deefeaa703bf51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.657809 20932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19179f35b2424ac997deefeaa703bf51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.658123 20933 tablet_service.cc:1514] Processing DeleteTablet for tablet 8480293812414b769cbf5bc3c20262aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:43 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:46224
I20260807 15:06:43.658329 20712 tablet_service.cc:1514] Processing DeleteTablet for tablet 8480293812414b769cbf5bc3c20262aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:43 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:39106
I20260807 15:06:43.658810 20712 tablet_service.cc:1467] Processing CreateTablet for tablet 19179f35b2424ac997deefeaa703bf51 (DEFAULT_TABLE table=test-table [id=45ecd47dc11b4bb296321310031f3578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:43.658944 20712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19179f35b2424ac997deefeaa703bf51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.659207 21285 tablet_bootstrap.cc:492] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: Bootstrap starting.
I20260807 15:06:43.659255 21277 tablet_bootstrap.cc:492] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: Bootstrap starting.
I20260807 15:06:43.659633 21277 tablet_bootstrap.cc:654] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.659633 21285 tablet_bootstrap.cc:654] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.659958 21279 tablet_bootstrap.cc:492] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: Bootstrap starting.
I20260807 15:06:43.660223 21311 tablet_replica.cc:330] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: stopping tablet replica
I20260807 15:06:43.660324 21279 tablet_bootstrap.cc:654] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.660337 21311 raft_consensus.cc:2229] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:43.660440 21311 raft_consensus.cc:2258] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:43.660597 21312 tablet_replica.cc:330] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: stopping tablet replica
I20260807 15:06:43.660696 21312 raft_consensus.cc:2229] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:43.660791 21312 raft_consensus.cc:2258] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:43.661207 21312 ts_tablet_manager.cc:1905] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:43.661207 21311 ts_tablet_manager.cc:1905] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:43.662596 21310 tablet_replica.cc:330] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: stopping tablet replica
I20260807 15:06:43.662732 21310 raft_consensus.cc:2229] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:06:43.662999 21311 ts_tablet_manager.cc:1918] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:43.663018 21312 ts_tablet_manager.cc:1918] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:43.663087 21312 log.cc:1196] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/8480293812414b769cbf5bc3c20262aa
I20260807 15:06:43.663087 21311 log.cc:1196] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/8480293812414b769cbf5bc3c20262aa
I20260807 15:06:43.663367 21312 ts_tablet_manager.cc:1939] T 8480293812414b769cbf5bc3c20262aa P 33bacb23b6e34f859b72dd547eeb04c9: Deleting consensus metadata
I20260807 15:06:43.663367 21311 ts_tablet_manager.cc:1939] T 8480293812414b769cbf5bc3c20262aa P 3a78167f087944a49f68d584decc98d7: Deleting consensus metadata
I20260807 15:06:43.663828 20495 catalog_manager.cc:4841] TS 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475): tablet 8480293812414b769cbf5bc3c20262aa (table test-table [id=6253400bdc0242299682a92f04e21215]) successfully deleted
I20260807 15:06:43.663839 20497 catalog_manager.cc:4841] TS 33bacb23b6e34f859b72dd547eeb04c9 (127.18.129.1:43895): tablet 8480293812414b769cbf5bc3c20262aa (table test-table [id=6253400bdc0242299682a92f04e21215]) successfully deleted
I20260807 15:06:43.666134 21310 raft_consensus.cc:2258] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:43.666584 21279 tablet_bootstrap.cc:492] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: No bootstrap required, opened a new log
I20260807 15:06:43.666630 21279 ts_tablet_manager.cc:1397] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260807 15:06:43.666628 21310 ts_tablet_manager.cc:1905] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:43.666966 21279 raft_consensus.cc:348] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.667055 21279 raft_consensus.cc:374] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.667083 21279 raft_consensus.cc:729] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33bacb23b6e34f859b72dd547eeb04c9, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.667124 21279 consensus_queue.cc:260] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.667232 21279 ts_tablet_manager.cc:1428] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:43.667599 21285 tablet_bootstrap.cc:492] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: No bootstrap required, opened a new log
I20260807 15:06:43.667649 21285 ts_tablet_manager.cc:1397] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260807 15:06:43.667791 21285 raft_consensus.cc:348] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.667860 21285 raft_consensus.cc:374] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.667877 21285 raft_consensus.cc:729] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a78167f087944a49f68d584decc98d7, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.667915 21285 consensus_queue.cc:260] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.668020 21285 ts_tablet_manager.cc:1428] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.668248 21310 ts_tablet_manager.cc:1918] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:43.668326 21310 log.cc:1196] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/8480293812414b769cbf5bc3c20262aa
I20260807 15:06:43.668500 21277 tablet_bootstrap.cc:492] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: No bootstrap required, opened a new log
I20260807 15:06:43.668545 21277 ts_tablet_manager.cc:1397] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260807 15:06:43.668581 21310 ts_tablet_manager.cc:1939] T 8480293812414b769cbf5bc3c20262aa P 52274734f1ac41109986701bfacfa3fc: Deleting consensus metadata
I20260807 15:06:43.668690 21277 raft_consensus.cc:348] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.668736 21277 raft_consensus.cc:374] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.668748 21277 raft_consensus.cc:729] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52274734f1ac41109986701bfacfa3fc, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.668787 21277 consensus_queue.cc:260] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.668864 21277 ts_tablet_manager.cc:1428] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.668939 20494 catalog_manager.cc:4841] TS 52274734f1ac41109986701bfacfa3fc (127.18.129.3:33127): tablet 8480293812414b769cbf5bc3c20262aa (table test-table [id=6253400bdc0242299682a92f04e21215]) successfully deleted
W20260807 15:06:43.678550 21234 tablet.cc:2369] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:43.835408 21304 raft_consensus.cc:482] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:43.835522 21304 raft_consensus.cc:504] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.835691 21304 leader_election.cc:288] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33bacb23b6e34f859b72dd547eeb04c9 (127.18.129.1:43895), 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475)
I20260807 15:06:43.835978 20964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19179f35b2424ac997deefeaa703bf51" candidate_uuid: "52274734f1ac41109986701bfacfa3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a78167f087944a49f68d584decc98d7" is_pre_election: true
I20260807 15:06:43.835975 20744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19179f35b2424ac997deefeaa703bf51" candidate_uuid: "52274734f1ac41109986701bfacfa3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33bacb23b6e34f859b72dd547eeb04c9" is_pre_election: true
I20260807 15:06:43.836093 20964 raft_consensus.cc:2454] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52274734f1ac41109986701bfacfa3fc in term 0.
I20260807 15:06:43.836091 20744 raft_consensus.cc:2454] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52274734f1ac41109986701bfacfa3fc in term 0.
I20260807 15:06:43.836268 21075 leader_election.cc:302] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a78167f087944a49f68d584decc98d7, 52274734f1ac41109986701bfacfa3fc; no voters:
I20260807 15:06:43.836391 21304 raft_consensus.cc:2781] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:43.836426 21304 raft_consensus.cc:482] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:43.836439 21304 raft_consensus.cc:3037] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.837168 21304 raft_consensus.cc:504] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.837287 21304 leader_election.cc:288] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [CANDIDATE]: Term 1 election: Requested vote from peers 33bacb23b6e34f859b72dd547eeb04c9 (127.18.129.1:43895), 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475)
I20260807 15:06:43.837486 20964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19179f35b2424ac997deefeaa703bf51" candidate_uuid: "52274734f1ac41109986701bfacfa3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a78167f087944a49f68d584decc98d7"
I20260807 15:06:43.837563 20964 raft_consensus.cc:3037] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.837554 20744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19179f35b2424ac997deefeaa703bf51" candidate_uuid: "52274734f1ac41109986701bfacfa3fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33bacb23b6e34f859b72dd547eeb04c9"
I20260807 15:06:43.837651 20744 raft_consensus.cc:3037] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.838163 20964 raft_consensus.cc:2454] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52274734f1ac41109986701bfacfa3fc in term 1.
I20260807 15:06:43.838323 21075 leader_election.cc:302] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a78167f087944a49f68d584decc98d7, 52274734f1ac41109986701bfacfa3fc; no voters:
I20260807 15:06:43.838390 20744 raft_consensus.cc:2454] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52274734f1ac41109986701bfacfa3fc in term 1.
I20260807 15:06:43.838418 21304 raft_consensus.cc:2781] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:43.838466 21304 raft_consensus.cc:686] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 1 LEADER]: Becoming Leader. State: Replica: 52274734f1ac41109986701bfacfa3fc, State: Running, Role: LEADER
I20260807 15:06:43.838511 21304 consensus_queue.cc:237] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.839130 20516 catalog_manager.cc:5495] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 52274734f1ac41109986701bfacfa3fc (127.18.129.3). New cstate: current_term: 1 leader_uuid: "52274734f1ac41109986701bfacfa3fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } health_report { overall_health: HEALTHY } } }
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115202, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:43.918186 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.911626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40825 (local address 127.0.0.1:46838)
0807 15:06:43.911775 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.911787 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.911869 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.912009 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.912010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.912427 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:43.912490 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.912493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.913150 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.913151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.913685 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.913687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.914013 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.914013 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.914073 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.914517 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:43.914521 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:43.915913 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:43.917124 (+ 1211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.917132 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.917135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.917343 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.917519 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:43.917520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:43.917520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:43.917578 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:43.917785 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:43.917793 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:43.917840 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.917899 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.918039 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":4,"thread_start_us":74,"threads_started":1}
I20260807 15:06:43.918186 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.911682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46838 (local address 127.18.129.62:40825)
0807 15:06:43.911748 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.911750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.911855 (+ 105us) server_negotiation.cc:407] Connection header received
0807 15:06:43.911930 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.911931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.911940 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.911995 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.912562 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.913080 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.913755 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.913958 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.914014 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.915992 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.915994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.915994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.916001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.917053 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.917409 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.917410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.917411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.917469 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.917626 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.917627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.917627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.917734 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.917790 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.917929 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.918092 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":22}
I20260807 15:06:43.920374 20516 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:46838:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:06:43.920701 20516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:43.922072 20712 tablet_service.cc:1467] Processing CreateTablet for tablet bdce1d4c15054c4f872361eff2eeb33f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc624b065a064bb2951765d208101f99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:06:43.922093 20932 tablet_service.cc:1467] Processing CreateTablet for tablet bdce1d4c15054c4f872361eff2eeb33f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc624b065a064bb2951765d208101f99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:06:43.922302 20932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdce1d4c15054c4f872361eff2eeb33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.922308 20712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdce1d4c15054c4f872361eff2eeb33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.923677 21285 tablet_bootstrap.cc:492] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: Bootstrap starting.
I20260807 15:06:43.923681 21279 tablet_bootstrap.cc:492] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: Bootstrap starting.
I20260807 15:06:43.923779 21141 tablet_service.cc:1467] Processing CreateTablet for tablet bdce1d4c15054c4f872361eff2eeb33f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc624b065a064bb2951765d208101f99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:06:43.923918 21141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdce1d4c15054c4f872361eff2eeb33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:43.924050 21285 tablet_bootstrap.cc:654] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.924106 21279 tablet_bootstrap.cc:654] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.924664 21285 tablet_bootstrap.cc:492] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: No bootstrap required, opened a new log
I20260807 15:06:43.924710 21285 ts_tablet_manager.cc:1397] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:43.924841 21285 raft_consensus.cc:348] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.924901 21285 raft_consensus.cc:374] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.924912 21285 raft_consensus.cc:729] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a78167f087944a49f68d584decc98d7, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.924947 21285 consensus_queue.cc:260] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.925030 21285 ts_tablet_manager.cc:1428] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.925096 21291 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.925171 21291 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:43.925256 21279 tablet_bootstrap.cc:492] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: No bootstrap required, opened a new log
I20260807 15:06:43.925298 21279 ts_tablet_manager.cc:1397] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:06:43.925315 21277 tablet_bootstrap.cc:492] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: Bootstrap starting.
I20260807 15:06:43.925431 21279 raft_consensus.cc:348] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.925486 21279 raft_consensus.cc:374] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.925498 21279 raft_consensus.cc:729] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33bacb23b6e34f859b72dd547eeb04c9, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.925529 21279 consensus_queue.cc:260] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.925606 21279 ts_tablet_manager.cc:1428] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.925638 21277 tablet_bootstrap.cc:654] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:43.925670 21287 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.925753 21287 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:43.926469 21277 tablet_bootstrap.cc:492] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: No bootstrap required, opened a new log
I20260807 15:06:43.926513 21277 ts_tablet_manager.cc:1397] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:43.926599 21277 raft_consensus.cc:348] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.926648 21277 raft_consensus.cc:374] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:43.926661 21277 raft_consensus.cc:729] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52274734f1ac41109986701bfacfa3fc, State: Initialized, Role: FOLLOWER
I20260807 15:06:43.926695 21277 consensus_queue.cc:260] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.926803 21277 ts_tablet_manager.cc:1428] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:43.926810 21304 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.927016 21304 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:43.928776 21287 raft_consensus.cc:482] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:43.928831 21287 raft_consensus.cc:504] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.929088 21287 leader_election.cc:288] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475), 52274734f1ac41109986701bfacfa3fc (127.18.129.3:33127)
I20260807 15:06:43.932210 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.929430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:51711 (local address 127.18.129.2:40475)
0807 15:06:43.929522 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.929524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.929547 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:06:43.929567 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.929568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.929577 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.929618 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:43.929788 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.930286 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.931503 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.931891 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.931953 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.931962 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.932113 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.932147 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 15:06:43.932224 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.929475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:44315 (local address 127.18.129.3:33127)
0807 15:06:43.929516 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.929517 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.929523 (+ 6us) server_negotiation.cc:407] Connection header received
0807 15:06:43.929542 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.929543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.929552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.929587 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:43.929731 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.930220 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.931510 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.931919 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.931978 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.931987 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.932125 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.932160 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:06:43.932271 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.929224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:33127 (local address 127.18.129.1:44315)
0807 15:06:43.929357 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.929365 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.929414 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.929599 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.929600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.929602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:43.929670 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.929672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.930294 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.930295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.931446 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.931449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.931987 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.931988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.932067 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.932071 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.932114 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.932158 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":48,"thread_start_us":97,"threads_started":1}
I20260807 15:06:43.932363 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.929208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:40475 (local address 127.18.129.1:51711)
0807 15:06:43.929304 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.929314 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.929515 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.929657 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.929658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.929659 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:43.929710 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.929711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.930356 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.930357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.931430 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.931433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.931974 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.931975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.932046 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.932049 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.932100 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.932146 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":27,"thread_start_us":48,"threads_started":1}
I20260807 15:06:43.932487 21164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdce1d4c15054c4f872361eff2eeb33f" candidate_uuid: "33bacb23b6e34f859b72dd547eeb04c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52274734f1ac41109986701bfacfa3fc" is_pre_election: true
I20260807 15:06:43.932592 21164 raft_consensus.cc:2454] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33bacb23b6e34f859b72dd547eeb04c9 in term 0.
I20260807 15:06:43.932659 20964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdce1d4c15054c4f872361eff2eeb33f" candidate_uuid: "33bacb23b6e34f859b72dd547eeb04c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a78167f087944a49f68d584decc98d7" is_pre_election: true
I20260807 15:06:43.932750 20964 raft_consensus.cc:2454] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33bacb23b6e34f859b72dd547eeb04c9 in term 0.
I20260807 15:06:43.932767 20649 leader_election.cc:302] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33bacb23b6e34f859b72dd547eeb04c9, 52274734f1ac41109986701bfacfa3fc; no voters:
I20260807 15:06:43.932883 21287 raft_consensus.cc:2781] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:43.932919 21287 raft_consensus.cc:482] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:43.932938 21287 raft_consensus.cc:3037] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.933516 21287 raft_consensus.cc:504] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.933637 21287 leader_election.cc:288] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [CANDIDATE]: Term 1 election: Requested vote from peers 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475), 52274734f1ac41109986701bfacfa3fc (127.18.129.3:33127)
I20260807 15:06:43.933815 20964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdce1d4c15054c4f872361eff2eeb33f" candidate_uuid: "33bacb23b6e34f859b72dd547eeb04c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a78167f087944a49f68d584decc98d7"
I20260807 15:06:43.933810 21164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdce1d4c15054c4f872361eff2eeb33f" candidate_uuid: "33bacb23b6e34f859b72dd547eeb04c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52274734f1ac41109986701bfacfa3fc"
I20260807 15:06:43.933890 20964 raft_consensus.cc:3037] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.933893 21164 raft_consensus.cc:3037] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:43.934439 20964 raft_consensus.cc:2454] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33bacb23b6e34f859b72dd547eeb04c9 in term 1.
I20260807 15:06:43.934480 21164 raft_consensus.cc:2454] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33bacb23b6e34f859b72dd547eeb04c9 in term 1.
I20260807 15:06:43.934597 20650 leader_election.cc:302] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33bacb23b6e34f859b72dd547eeb04c9, 3a78167f087944a49f68d584decc98d7; no voters:
I20260807 15:06:43.934684 21287 raft_consensus.cc:2781] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:43.934870 21287 raft_consensus.cc:686] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 LEADER]: Becoming Leader. State: Replica: 33bacb23b6e34f859b72dd547eeb04c9, State: Running, Role: LEADER
I20260807 15:06:43.934963 21287 consensus_queue.cc:237] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } }
I20260807 15:06:43.935253 21336 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: TxnStatusTablet state changed. Reason: New leader 33bacb23b6e34f859b72dd547eeb04c9. Latest consensus state: current_term: 1 leader_uuid: "33bacb23b6e34f859b72dd547eeb04c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.935338 21336 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:06:43.935520 21339 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:06:43.935628 21339 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:06:43.935549 20516 catalog_manager.cc:5495] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33bacb23b6e34f859b72dd547eeb04c9 (127.18.129.1). New cstate: current_term: 1 leader_uuid: "33bacb23b6e34f859b72dd547eeb04c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } health_report { overall_health: UNKNOWN } } }
I20260807 15:06:43.943550 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.940969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39120 (local address 127.18.129.1:43895)
0807 15:06:43.941013 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.941014 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.941097 (+ 83us) server_negotiation.cc:407] Connection header received
0807 15:06:43.941191 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.941192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.941206 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.941257 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:43.941367 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.941797 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.942828 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.943054 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.943106 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.943291 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:43.943375 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:43.943423 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.943488 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.943502 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:06:43.943619 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.940909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:43895 (local address 127.0.0.1:39120)
0807 15:06:43.941045 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.941057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.941121 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.941263 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.941263 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.941265 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:06:43.941313 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.941316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.941863 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.941864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.942754 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.942757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.943125 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.943125 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.943184 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.943202 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:06:43.943426 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:06:43.943433 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.943485 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.943541 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 15:06:43.944646 20964 raft_consensus.cc:1261] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Refusing update from remote peer 33bacb23b6e34f859b72dd547eeb04c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:43.944675 21164 raft_consensus.cc:1261] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc [term 1 FOLLOWER]: Refusing update from remote peer 33bacb23b6e34f859b72dd547eeb04c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:43.944898 21336 consensus_queue.cc:1035] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:43.945019 21287 consensus_queue.cc:1035] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:43.945583 21304 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: TxnStatusTablet state changed. Reason: New leader 33bacb23b6e34f859b72dd547eeb04c9. Latest consensus state: current_term: 1 leader_uuid: "33bacb23b6e34f859b72dd547eeb04c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.945569 21291 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: TxnStatusTablet state changed. Reason: New leader 33bacb23b6e34f859b72dd547eeb04c9. Latest consensus state: current_term: 1 leader_uuid: "33bacb23b6e34f859b72dd547eeb04c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.945654 21304 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:43.945657 21291 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:43.946511 21291 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33bacb23b6e34f859b72dd547eeb04c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.946606 21291 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 3a78167f087944a49f68d584decc98d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:43.946426 21287 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33bacb23b6e34f859b72dd547eeb04c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.947034 21287 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:06:43.947116 21287 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33bacb23b6e34f859b72dd547eeb04c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.947166 21287 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 33bacb23b6e34f859b72dd547eeb04c9: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:06:43.947535 21304 tablet_replica.cc:439] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33bacb23b6e34f859b72dd547eeb04c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 } } peers { permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 } } peers { permanent_uuid: "52274734f1ac41109986701bfacfa3fc" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33127 } } }
I20260807 15:06:43.947607 21304 tablet_replica.cc:442] T bdce1d4c15054c4f872361eff2eeb33f P 52274734f1ac41109986701bfacfa3fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:43.953255 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.951082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:33041 (local address 127.18.129.1:43895)
0807 15:06:43.951117 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.951119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.951296 (+ 177us) server_negotiation.cc:407] Connection header received
0807 15:06:43.951344 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.951345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.951354 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.951403 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:43.951497 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.951952 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.952661 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.952854 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.952905 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.953056 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:43.953086 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:43.953124 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.953174 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.953191 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260807 15:06:43.953280 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.951037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:43895 (local address 127.18.129.3:33041)
0807 15:06:43.951182 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.951191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.951299 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.951407 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.951408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.951409 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:06:43.951454 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.951456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.952018 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.952019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.952593 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.952595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.952912 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.952912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.952968 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.952984 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:06:43.953131 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:06:43.953133 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.953173 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.953215 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":53,"threads_started":1}
I20260807 15:06:43.955386 20964 raft_consensus.cc:1261] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Refusing update from remote peer 52274734f1ac41109986701bfacfa3fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:43.955401 20744 raft_consensus.cc:1261] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Refusing update from remote peer 52274734f1ac41109986701bfacfa3fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:43.955597 21304 consensus_queue.cc:1035] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a78167f087944a49f68d584decc98d7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:43.955701 21289 consensus_queue.cc:1035] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "33bacb23b6e34f859b72dd547eeb04c9" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 43895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:43.964319 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.961518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:53581 (local address 127.18.129.3:33127)
0807 15:06:43.961564 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.961566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.961684 (+ 118us) server_negotiation.cc:407] Connection header received
0807 15:06:43.961748 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.961749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.961759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.961798 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:43.961942 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.962372 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.963573 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.963866 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.963903 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.964136 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:43.964165 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:43.964199 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.964255 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.964272 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:43.964483 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.961463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:33127 (local address 127.18.129.1:53581)
0807 15:06:43.961621 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:43.961632 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:43.961692 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:43.961808 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:43.961809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:43.961821 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:06:43.961895 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.961897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.962440 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.962441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.963458 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:43.963461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.963972 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.963973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:43.964035 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.964053 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:06:43.964206 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:06:43.964207 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:06:43.964252 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:06:43.964294 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260807 15:06:43.972544 20515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:43.972647 20515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:43.973198 20515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44b59e5da93f4cd7bc03c64d0e12c9aa: Sending DeleteTablet for 3 replicas of tablet 19179f35b2424ac997deefeaa703bf51
I20260807 15:06:43.973431 21141 tablet_service.cc:1514] Processing DeleteTablet for tablet 19179f35b2424ac997deefeaa703bf51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:43 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:52410
I20260807 15:06:43.973471 20932 tablet_service.cc:1514] Processing DeleteTablet for tablet 19179f35b2424ac997deefeaa703bf51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:43 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:46224
I20260807 15:06:43.973523 21310 tablet_replica.cc:330] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: stopping tablet replica
I20260807 15:06:43.973503 20712 tablet_service.cc:1514] Processing DeleteTablet for tablet 19179f35b2424ac997deefeaa703bf51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:43 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:39106
I20260807 15:06:43.973552 21311 tablet_replica.cc:330] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: stopping tablet replica
I20260807 15:06:43.973585 21311 raft_consensus.cc:2229] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:43.973583 21310 raft_consensus.cc:2229] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:06:43.973616 21311 raft_consensus.cc:2258] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:43.973631 21310 raft_consensus.cc:2258] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:43.973685 21312 tablet_replica.cc:330] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: stopping tablet replica
I20260807 15:06:43.973740 21312 raft_consensus.cc:2229] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:43.973781 21312 raft_consensus.cc:2258] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:43.973922 21310 ts_tablet_manager.cc:1905] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:43.974079 21311 ts_tablet_manager.cc:1905] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:43.974152 21312 ts_tablet_manager.cc:1905] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:43.975514 21311 ts_tablet_manager.cc:1918] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:43.975567 21311 log.cc:1196] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/19179f35b2424ac997deefeaa703bf51
I20260807 15:06:43.975566 21312 ts_tablet_manager.cc:1918] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:43.975566 21310 ts_tablet_manager.cc:1918] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:43.975617 21312 log.cc:1196] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/19179f35b2424ac997deefeaa703bf51
I20260807 15:06:43.975629 21310 log.cc:1196] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/19179f35b2424ac997deefeaa703bf51
I20260807 15:06:43.975777 21311 ts_tablet_manager.cc:1939] T 19179f35b2424ac997deefeaa703bf51 P 3a78167f087944a49f68d584decc98d7: Deleting consensus metadata
I20260807 15:06:43.975833 21312 ts_tablet_manager.cc:1939] T 19179f35b2424ac997deefeaa703bf51 P 33bacb23b6e34f859b72dd547eeb04c9: Deleting consensus metadata
I20260807 15:06:43.975832 21310 ts_tablet_manager.cc:1939] T 19179f35b2424ac997deefeaa703bf51 P 52274734f1ac41109986701bfacfa3fc: Deleting consensus metadata
I20260807 15:06:43.976074 20495 catalog_manager.cc:4841] TS 3a78167f087944a49f68d584decc98d7 (127.18.129.2:40475): tablet 19179f35b2424ac997deefeaa703bf51 (table test-table [id=45ecd47dc11b4bb296321310031f3578]) successfully deleted
I20260807 15:06:43.976276 20497 catalog_manager.cc:4841] TS 33bacb23b6e34f859b72dd547eeb04c9 (127.18.129.1:43895): tablet 19179f35b2424ac997deefeaa703bf51 (table test-table [id=45ecd47dc11b4bb296321310031f3578]) successfully deleted
I20260807 15:06:43.976382 20494 catalog_manager.cc:4841] TS 52274734f1ac41109986701bfacfa3fc (127.18.129.3:33127): tablet 19179f35b2424ac997deefeaa703bf51 (table test-table [id=45ecd47dc11b4bb296321310031f3578]) successfully deleted
Aug 07 15:06:43 dist-test-slave-5bn2 krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115203, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:06:43.987692 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.979110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.18.129.1:43895)
0807 15:06:43.979159 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.979161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.979277 (+ 116us) server_negotiation.cc:407] Connection header received
0807 15:06:43.979349 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.979350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.979358 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.979401 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.980089 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.980528 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.981557 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.981786 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.981832 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.985049 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.985058 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.985058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.985065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.986582 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.986949 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.986953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.986954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.987010 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.987190 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.987191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.987191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.987337 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.987486 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.987525 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.987597 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 15:06:43.987986 20801 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39132
I20260807 15:06:43.995005 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:43.988885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.18.129.62:40825)
0807 15:06:43.988929 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:43.988931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:43.988940 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:06:43.988967 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:43.988967 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:43.988977 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:43.989041 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:43.989459 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.990239 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.990870 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:43.991172 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:43.991239 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:43.992269 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:43.992271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:43.992271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:43.992278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:43.993957 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.994262 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.994263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.994263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.994324 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:43.994469 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:43.994469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:43.994470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:43.994606 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:43.994698 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:43.994811 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 15:06:43.994910 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 15:06:43.995182 20515 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46852
I20260807 15:06:43.995855 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20624
I20260807 15:06:44.005718 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20826
I20260807 15:06:44.023944 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 21049
I20260807 15:06:44.031594 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20475
2026-08-07T15:06:44Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2531 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:44 dist-test-slave-5bn2 krb5kdc[21371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:44 dist-test-slave-5bn2 krb5kdc[21371](info): set up 2 sockets
Aug 07 15:06:44 dist-test-slave-5bn2 krb5kdc[21371](info): commencing operation
krb5kdc: starting...
W20260807 15:06:45.159925 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:45.279372 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46375
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:45.360989 21390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:45.361214 21390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:45.361245 21390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:45.362372 21390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:45.362412 21390 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:06:45.362429 21390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:45.362445 21390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:45.362461 21390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:45.362484 21390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:45.364048 21390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46375
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:45.364300 21390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:45.364504 21390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:45.369306 21398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:45.369306 21397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:45.369303 21402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:45.369544 21390 server_base.cc:1034] running on GCE node
I20260807 15:06:45.369799 21390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:45.370067 21390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:45.371212 21390 hybrid_clock.cc:648] HybridClock initialized: now 1786115205371200 us; error 29 us; skew 500 ppm
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:45.373549 21390 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:45.373956 21390 webserver.cc:460] Webserver started at http://127.18.129.62:36309/ using document root <none> and password file <none>
I20260807 15:06:45.374150 21390 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:45.374194 21390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:45.374322 21390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:45.375479 21390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "6808e678c19d495b80c3d8a5b6aed517"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "6d553e7e381bedcd54a6d9386356e2c2"
server_key_iv: "7f7b29e0e7611b665286d2e4eb222318"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.375825 21390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "6808e678c19d495b80c3d8a5b6aed517"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "6d553e7e381bedcd54a6d9386356e2c2"
server_key_iv: "7f7b29e0e7611b665286d2e4eb222318"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.377025 21390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:06:45.377674 21414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:45.377828 21390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:45.377913 21390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "6808e678c19d495b80c3d8a5b6aed517"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "6d553e7e381bedcd54a6d9386356e2c2"
server_key_iv: "7f7b29e0e7611b665286d2e4eb222318"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.377974 21390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:45.397276 21390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:45.399017 21390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:45.399155 21390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:45.403609 21390 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:46375
I20260807 15:06:45.403683 21519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:46375 every 8 connection(s)
I20260807 15:06:45.403982 21390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:45.404423 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 21390
I20260807 15:06:45.404522 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:45.404722 18948 external_mini_cluster.cc:1428] Setting key 477f14541231c7e77e8cf312497cc8e8
I20260807 15:06:45.404731 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:45.407828 21520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: Bootstrap starting.
I20260807 15:06:45.408420 21520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:45.408694 21520 log.cc:824] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:45.409379 21520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: No bootstrap required, opened a new log
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:45.411535 21520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6808e678c19d495b80c3d8a5b6aed517" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46375 } }
I20260807 15:06:45.411697 21520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:45.411743 21520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6808e678c19d495b80c3d8a5b6aed517, State: Initialized, Role: FOLLOWER
I20260807 15:06:45.411881 21520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6808e678c19d495b80c3d8a5b6aed517" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46375 } }
I20260807 15:06:45.411957 21520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:45.411979 21520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:45.412027 21520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:45.412639 21520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6808e678c19d495b80c3d8a5b6aed517" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46375 } }
I20260807 15:06:45.412743 21520 leader_election.cc:302] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6808e678c19d495b80c3d8a5b6aed517; no voters:
I20260807 15:06:45.412895 21520 leader_election.cc:288] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:45.412945 21526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:45.413100 21526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [term 1 LEADER]: Becoming Leader. State: Replica: 6808e678c19d495b80c3d8a5b6aed517, State: Running, Role: LEADER
I20260807 15:06:45.413165 21520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:45.413208 21526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6808e678c19d495b80c3d8a5b6aed517" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46375 } }
I20260807 15:06:45.413597 21526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6808e678c19d495b80c3d8a5b6aed517. Latest consensus state: current_term: 1 leader_uuid: "6808e678c19d495b80c3d8a5b6aed517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6808e678c19d495b80c3d8a5b6aed517" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46375 } } }
I20260807 15:06:45.413739 21526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:45.413753 21527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6808e678c19d495b80c3d8a5b6aed517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6808e678c19d495b80c3d8a5b6aed517" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46375 } } }
I20260807 15:06:45.414286 21527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:45.415206 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.405228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.18.129.62:46375)
0807 15:06:45.405410 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.405418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.405443 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:06:45.405857 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.405864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.405910 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.406039 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:45.406655 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.407601 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.408364 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.408828 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.408923 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.411381 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:45.411385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:45.411386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:45.411398 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:45.413202 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.413538 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.413541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.413541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.413654 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.413862 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.413863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.413863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.414196 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:45.414513 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.414632 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.414764 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
W20260807 15:06:45.415395 21546 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:06:45.415482 21546 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:06:45.415731 21539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:45.415902 21539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:45.417225 21539 catalog_manager.cc:1347] Generated new cluster ID: 7d32e72daa634bea82d4093b652b8c46
I20260807 15:06:45.417279 21539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:45.431989 21539 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:45.432684 21539 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:45.435602 21539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: Generated new TSK 0
I20260807 15:06:45.435774 21539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:06:45.467892 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46375
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:45.548837 21551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:45.549052 21551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:45.549085 21551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:45.550225 21551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:45.550267 21551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:45.550294 21551 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:06:45.550309 21551 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:06:45.550333 21551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:45.551818 21551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:46375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:45.552048 21551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:45.552255 21551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:45.557556 21551 server_base.cc:1034] running on GCE node
W20260807 15:06:45.557503 21560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:45.557487 21557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:45.557518 21558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:45.557958 21551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:45.558212 21551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:45.559343 21551 hybrid_clock.cc:648] HybridClock initialized: now 1786115205559335 us; error 27 us; skew 500 ppm
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:45.561726 21551 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:06:45.562158 21551 webserver.cc:460] Webserver started at http://127.18.129.1:44965/ using document root <none> and password file <none>
I20260807 15:06:45.562359 21551 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:45.562408 21551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:45.562537 21551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:45.563642 21551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "fa18c579e45c4a9588bf4427b78f1156"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "b8658a44d2628b7f034f4c851a62163f"
server_key_iv: "a5f56e6ddef2c6970df72f4218738e28"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.563987 21551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "fa18c579e45c4a9588bf4427b78f1156"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "b8658a44d2628b7f034f4c851a62163f"
server_key_iv: "a5f56e6ddef2c6970df72f4218738e28"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.565227 21551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:06:45.565903 21569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:45.566036 21551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:06:45.566129 21551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "fa18c579e45c4a9588bf4427b78f1156"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "b8658a44d2628b7f034f4c851a62163f"
server_key_iv: "a5f56e6ddef2c6970df72f4218738e28"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.566195 21551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:45.587687 21551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:45.589504 21551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:45.589645 21551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:45.590193 21551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:45.590236 21551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:45.590267 21551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:45.590286 21551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:45.597353 21551 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:35385
I20260807 15:06:45.597450 21746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:35385 every 8 connection(s)
I20260807 15:06:45.597735 21551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:06:45.600369 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.590669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:46437 (local address 127.18.129.62:46375)
0807 15:06:45.590721 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.590724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.590988 (+ 264us) server_negotiation.cc:407] Connection header received
0807 15:06:45.591504 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.591506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.591519 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.591573 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:45.592253 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.592772 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.594032 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.594268 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.594328 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.597637 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:45.597639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:45.597640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:45.597652 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:45.599059 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.599448 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.599449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.599449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.599513 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.599763 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.599764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.599765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.599924 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:45.599991 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.600193 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.600294 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:06:45.600464 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.590665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46375 (local address 127.18.129.1:46437)
0807 15:06:45.590832 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.590905 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.591407 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.591647 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.591651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.591952 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:45.592151 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.592156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.592857 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.592858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.593921 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.593925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.594346 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.594347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.594503 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.595118 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:45.595130 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:45.597527 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:45.599130 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.599135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.599136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.599351 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.599570 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.599570 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.599570 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.599667 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.599983 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:45.599986 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:45.600097 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.600170 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.600291 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260807 15:06:45.600980 21749 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46375
I20260807 15:06:45.601056 21749 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:45.601219 21749 heartbeater.cc:511] Master 127.18.129.62:46375 requested a full tablet report, sending...
I20260807 15:06:45.601740 21435 ts_manager.cc:194] Registered new tserver with Master: fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385)
I20260807 15:06:45.602571 21435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:46437
I20260807 15:06:45.603318 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 21551
I20260807 15:06:45.603423 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:45.603624 18948 external_mini_cluster.cc:1428] Setting key 924fa06ef848a155296566af30483c15
I20260807 15:06:45.607586 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.602112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:40325 (local address 127.18.129.62:46375)
0807 15:06:45.602155 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.602157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.602218 (+ 61us) server_negotiation.cc:407] Connection header received
0807 15:06:45.602281 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.602281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.602289 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.602337 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:45.602674 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.603287 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.604190 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.604406 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.604468 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.605314 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:45.605316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:45.605316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:45.605323 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:45.606566 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.606885 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.606886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.606887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.606942 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.607107 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.607108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.607108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.607210 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:45.607264 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.607405 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.607483 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:06:45.607594 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.602037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46375 (local address 127.18.129.1:40325)
0807 15:06:45.602161 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.602172 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.602230 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.602351 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.602352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.602538 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:45.602607 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.602608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.603466 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.603467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.604102 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.604105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.604479 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.604480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.604557 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.604936 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:45.604941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:45.605233 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:45.606644 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.606645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.606645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.606805 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.606996 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.606997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.606997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.607058 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.607262 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:45.607263 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:45.607308 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.607377 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.607485 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:06:45.629217 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46375
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:45.710843 21765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:45.711087 21765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:45.711109 21765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:45.712222 21765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:45.712257 21765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:45.712276 21765 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:06:45.712299 21765 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:06:45.712314 21765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:45.713824 21765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:46375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:45.714040 21765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:45.714233 21765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:45.719241 21776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:45.719254 21765 server_base.cc:1034] running on GCE node
W20260807 15:06:45.719213 21785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:45.719220 21775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:45.719560 21765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:45.719769 21765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:45.720904 21765 hybrid_clock.cc:648] HybridClock initialized: now 1786115205720892 us; error 31 us; skew 500 ppm
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:45.723381 21765 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:06:45.723779 21765 webserver.cc:460] Webserver started at http://127.18.129.2:40917/ using document root <none> and password file <none>
I20260807 15:06:45.724001 21765 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:45.724052 21765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:45.724185 21765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:45.725217 21765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "636960a0dafc4904a58bb65895f443b7"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "02691df72697ea124c07e6fad4eac433"
server_key_iv: "a3721b18789d936f36a7a56e2f072aa1"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.725543 21765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "636960a0dafc4904a58bb65895f443b7"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "02691df72697ea124c07e6fad4eac433"
server_key_iv: "a3721b18789d936f36a7a56e2f072aa1"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.726776 21765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:06:45.727483 21797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:45.727631 21765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:45.727715 21765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "636960a0dafc4904a58bb65895f443b7"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "02691df72697ea124c07e6fad4eac433"
server_key_iv: "a3721b18789d936f36a7a56e2f072aa1"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.727777 21765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:45.740669 21765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:45.742745 21765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:45.742905 21765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:45.743393 21765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:45.743432 21765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:45.743460 21765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:45.743479 21765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:45.750483 21765 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:40623
I20260807 15:06:45.750609 21956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:40623 every 8 connection(s)
I20260807 15:06:45.750887 21765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:06:45.752792 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.743825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:54593 (local address 127.18.129.62:46375)
0807 15:06:45.743890 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.743893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.744078 (+ 185us) server_negotiation.cc:407] Connection header received
0807 15:06:45.744540 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.744542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.744556 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.744643 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:45.745408 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.746148 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.747413 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.747608 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.747665 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.749943 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:45.749945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:45.749945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:45.749954 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:45.751515 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.751905 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.751906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.751906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.751964 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.752190 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.752191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.752191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.752308 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:45.752364 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.752617 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.752699 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:06:45.752947 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.743776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46375 (local address 127.18.129.2:54593)
0807 15:06:45.743942 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.743981 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.744453 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.744717 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.744721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.745026 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:45.745304 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.745311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.746246 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.746247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.747323 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.747326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.747693 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.747694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.747825 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.748318 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:45.748328 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:45.749865 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:45.751596 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.751602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.751604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.751837 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.752018 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.752019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.752019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.752132 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.752362 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:45.752369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:45.752497 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.752586 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.752740 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260807 15:06:45.753408 21957 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46375
I20260807 15:06:45.753495 21957 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:45.753671 21957 heartbeater.cc:511] Master 127.18.129.62:46375 requested a full tablet report, sending...
I20260807 15:06:45.754002 21437 ts_manager.cc:194] Registered new tserver with Master: 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623)
I20260807 15:06:45.754233 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 21765
I20260807 15:06:45.754335 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:45.754498 18948 external_mini_cluster.cc:1428] Setting key 284337dd0cbdc038662dccd0fec0ee19
I20260807 15:06:45.754503 21437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:54593
I20260807 15:06:45.759670 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.754398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46375 (local address 127.18.129.2:48891)
0807 15:06:45.754551 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.754560 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.754612 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.754733 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.754734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.754947 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:45.755004 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.755005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.755742 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.755743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.756218 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.756220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.756596 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.756597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.756648 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.757002 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:45.757006 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:45.757269 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:45.758719 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.758720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.758721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.758912 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.759102 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.759102 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.759103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.759155 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.759446 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:45.759447 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:45.759481 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.759523 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.759605 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260807 15:06:45.759716 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.754452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:48891 (local address 127.18.129.62:46375)
0807 15:06:45.754491 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.754492 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.754615 (+ 123us) server_negotiation.cc:407] Connection header received
0807 15:06:45.754665 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.754666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.754676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.754719 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:45.755063 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.755646 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.756289 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.756530 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.756595 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.757346 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:45.757347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:45.757348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:45.757355 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:45.758645 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.758974 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.758975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.758975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.759044 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.759199 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.759199 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.759199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.759378 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:45.759456 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.759555 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.759630 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:06:45.779431 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46375
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:45.860033 21974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:45.860245 21974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:45.860270 21974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:45.861377 21974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:45.861414 21974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:45.861433 21974 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:06:45.861447 21974 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:06:45.861464 21974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:45.862942 21974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:46375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:45.863137 21974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:45.863327 21974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:45.868340 21983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:45.868309 21986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:45.868316 21982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:45.868392 21974 server_base.cc:1034] running on GCE node
I20260807 15:06:45.868661 21974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:45.868913 21974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:45.870059 21974 hybrid_clock.cc:648] HybridClock initialized: now 1786115205870037 us; error 36 us; skew 500 ppm
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:45.872500 21974 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:06:45.872910 21974 webserver.cc:460] Webserver started at http://127.18.129.3:42735/ using document root <none> and password file <none>
I20260807 15:06:45.873135 21974 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:45.873185 21974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:45.873310 21974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:45.874361 21974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "781c2914a85f45ffa85c6f4ed1ae7faf"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "370bb1824260eb8c50a1bcb96493d82d"
server_key_iv: "fe2849cf285224b7ca963381942b2ce8"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.874701 21974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "781c2914a85f45ffa85c6f4ed1ae7faf"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "370bb1824260eb8c50a1bcb96493d82d"
server_key_iv: "fe2849cf285224b7ca963381942b2ce8"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.875982 21974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:45.876637 21994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:45.876791 21974 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:06:45.876865 21974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "781c2914a85f45ffa85c6f4ed1ae7faf"
format_stamp: "Formatted at 2026-08-07 15:06:45 on dist-test-slave-5bn2"
server_key: "370bb1824260eb8c50a1bcb96493d82d"
server_key_iv: "fe2849cf285224b7ca963381942b2ce8"
server_key_version: "encryptionkey@0"
I20260807 15:06:45.876933 21974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:45.900138 21974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:45.902033 21974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:45.902201 21974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:45.902732 21974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:45.902777 21974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:45.902810 21974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:45.902827 21974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:45.909824 21974 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:42711
I20260807 15:06:45.909888 22173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:42711 every 8 connection(s)
I20260807 15:06:45.910287 21974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:06:45.912603 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.903196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:59065 (local address 127.18.129.62:46375)
0807 15:06:45.903263 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.903266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.903488 (+ 222us) server_negotiation.cc:407] Connection header received
0807 15:06:45.904198 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.904200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.904216 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.904280 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:45.905194 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.905959 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.907458 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.907716 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.907782 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.910127 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:45.910129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:45.910130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:45.910138 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:45.911556 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.911906 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.911907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.911907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.911955 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.912139 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.912140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.912140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.912235 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:45.912280 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.912477 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.912535 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:06:45.912796 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.903172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46375 (local address 127.18.129.3:59065)
0807 15:06:45.903342 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.903395 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.904069 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.904370 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.904374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.904781 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:45.905091 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.905098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.906051 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.906052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.907369 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.907372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.907810 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.907811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.907933 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.908397 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:45.908405 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:45.910019 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:45.911625 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.911631 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.911632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.911847 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.911996 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.911997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.911997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.912097 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.912276 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:45.912279 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:45.912389 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.912463 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.912584 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260807 15:06:45.913316 22178 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46375
I20260807 15:06:45.913407 22178 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:45.913589 22178 heartbeater.cc:511] Master 127.18.129.62:46375 requested a full tablet report, sending...
I20260807 15:06:45.914011 21435 ts_manager.cc:194] Registered new tserver with Master: 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:45.914165 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 21974
I20260807 15:06:45.914263 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:45.914467 18948 external_mini_cluster.cc:1428] Setting key 1d219ba8684ac1a67a8b96934eb9f207
I20260807 15:06:45.914528 21435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:59065
I20260807 15:06:45.915812 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:06:45.920507 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.914388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46375 (local address 127.18.129.3:41063)
0807 15:06:45.915457 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.915468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.915521 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.915662 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.915663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.915862 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:45.915910 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.915912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.916533 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.916534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.917017 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.917018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.917339 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.917340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.917415 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.917830 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:45.917835 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:45.918113 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:45.919641 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.919642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.919642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.919803 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.919986 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:45.919986 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:45.919987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:45.920040 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:45.920286 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:45.920286 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:45.920321 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.920365 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.920439 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"thread_start_us":83,"threads_started":1}
I20260807 15:06:45.920557 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.914440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:41063 (local address 127.18.129.62:46375)
0807 15:06:45.914722 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.914724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.915512 (+ 788us) server_negotiation.cc:407] Connection header received
0807 15:06:45.915593 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.915594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.915604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.915651 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:45.915977 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.916447 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.917083 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.917281 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.917334 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.918194 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:45.918195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:45.918196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:45.918203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:45.919557 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.919867 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.919868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.919869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.919932 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.920089 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.920090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.920090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.920218 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:45.920295 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.920393 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.920469 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:06:45 dist-test-slave-5bn2 krb5kdc[21371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:45.939282 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.932857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51366 (local address 127.18.129.62:46375)
0807 15:06:45.932912 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.932913 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.932923 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:45.932979 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.932980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.932989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.933038 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:45.933569 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.934050 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.934764 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.935026 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.935089 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.937164 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:45.937165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:45.937165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:45.937173 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:45.938257 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.938554 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.938555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.938556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.938609 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:45.938767 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:45.938768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:45.938768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:45.938901 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:45.938988 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.939115 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.939190 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:06:45.940680 21435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:45.942217 21435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:45.948858 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.944497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.18.129.3:42711)
0807 15:06:45.945101 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.945106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.945116 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:45.945156 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.945158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.945173 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.945251 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:45.945475 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.946239 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.947767 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.948444 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.948502 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.948562 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.948715 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.948757 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":98,"threads_started":1}
I20260807 15:06:45.948853 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.944371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:40623 (local address 127.0.0.1:38032)
0807 15:06:45.944539 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.944584 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.944651 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.945513 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.945513 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.945515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:45.945566 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.945567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.946675 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.946676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.947587 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.947590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.948498 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.948498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.948601 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.948619 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.948690 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.948755 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":23,"thread_start_us":49,"threads_started":1}
I20260807 15:06:45.948853 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.944390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:42711 (local address 127.0.0.1:53932)
0807 15:06:45.944684 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.944694 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.944745 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.945345 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.945346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.945348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:45.945406 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.945407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.946314 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.946315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.947694 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.947696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.948519 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.948520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.948601 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.948619 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.948690 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.948756 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":33,"thread_start_us":89,"threads_started":1}
I20260807 15:06:45.949129 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.944578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38032 (local address 127.18.129.2:40623)
0807 15:06:45.945377 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.945381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.945389 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:45.945425 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.945427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.945442 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.945516 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:45.945625 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.946606 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.947665 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.948417 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.948498 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.948563 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.948981 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.949023 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":97,"threads_started":1}
I20260807 15:06:45.949755 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.944558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35385 (local address 127.0.0.1:41908)
0807 15:06:45.944921 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:45.944929 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:45.944972 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:45.945145 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:45.945149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:45.945152 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:45.945229 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.945231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.946474 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.946475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.947688 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:45.947692 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.949522 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.949523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:45.949606 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.949612 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:06:45.949659 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:06:45.949700 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":9,"thread_start_us":65,"threads_started":1}
I20260807 15:06:45.949827 22074 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8e60a8d38b4402a62fb781261a0bfe (DEFAULT_TABLE table=test-table [id=92c270720a0042d1a7d18d0ade24457a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:45.949826 21867 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8e60a8d38b4402a62fb781261a0bfe (DEFAULT_TABLE table=test-table [id=92c270720a0042d1a7d18d0ade24457a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:45.950052 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:45.944776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.18.129.1:35385)
0807 15:06:45.944935 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:45.944939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:45.944965 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:06:45.945026 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:45.945027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:45.945049 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:45.945120 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:45.945298 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.946394 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.948611 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:45.949361 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:45.949428 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:45.949490 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:45.949915 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 15:06:45.949956 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260807 15:06:45.950214 21867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8e60a8d38b4402a62fb781261a0bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:45.950214 22074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8e60a8d38b4402a62fb781261a0bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:45.950600 21654 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8e60a8d38b4402a62fb781261a0bfe (DEFAULT_TABLE table=test-table [id=92c270720a0042d1a7d18d0ade24457a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:45.950963 21654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8e60a8d38b4402a62fb781261a0bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:45.953584 22213 tablet_bootstrap.cc:492] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Bootstrap starting.
I20260807 15:06:45.953919 22214 tablet_bootstrap.cc:492] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Bootstrap starting.
I20260807 15:06:45.954279 22215 tablet_bootstrap.cc:492] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Bootstrap starting.
I20260807 15:06:45.954663 22214 tablet_bootstrap.cc:654] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:45.954689 22213 tablet_bootstrap.cc:654] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:45.954973 22215 tablet_bootstrap.cc:654] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:45.954994 22214 log.cc:824] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:45.955039 22213 log.cc:824] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:45.955302 22215 log.cc:824] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:45.955724 22214 tablet_bootstrap.cc:492] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: No bootstrap required, opened a new log
I20260807 15:06:45.955801 22214 ts_tablet_manager.cc:1397] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:06:45.955893 22213 tablet_bootstrap.cc:492] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: No bootstrap required, opened a new log
I20260807 15:06:45.956013 22213 ts_tablet_manager.cc:1397] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 15:06:45.956213 22215 tablet_bootstrap.cc:492] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: No bootstrap required, opened a new log
I20260807 15:06:45.956283 22215 ts_tablet_manager.cc:1397] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:06:45.957233 22214 raft_consensus.cc:348] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:45.957366 22214 raft_consensus.cc:374] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:45.957397 22214 raft_consensus.cc:729] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa18c579e45c4a9588bf4427b78f1156, State: Initialized, Role: FOLLOWER
I20260807 15:06:45.957511 22214 consensus_queue.cc:260] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:45.957772 22214 ts_tablet_manager.cc:1428] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:45.957813 21749 heartbeater.cc:503] Master 127.18.129.62:46375 was elected leader, sending a full tablet report...
I20260807 15:06:45.958132 22215 raft_consensus.cc:348] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:45.958287 22215 raft_consensus.cc:374] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:45.958329 22215 raft_consensus.cc:729] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 636960a0dafc4904a58bb65895f443b7, State: Initialized, Role: FOLLOWER
I20260807 15:06:45.958364 22213 raft_consensus.cc:348] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:45.958439 22215 consensus_queue.cc:260] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:45.958503 22213 raft_consensus.cc:374] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:45.958544 22213 raft_consensus.cc:729] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781c2914a85f45ffa85c6f4ed1ae7faf, State: Initialized, Role: FOLLOWER
I20260807 15:06:45.958702 22215 ts_tablet_manager.cc:1428] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:06:45.958726 21957 heartbeater.cc:503] Master 127.18.129.62:46375 was elected leader, sending a full tablet report...
I20260807 15:06:45.958665 22213 consensus_queue.cc:260] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:45.959204 22213 ts_tablet_manager.cc:1428] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:06:45.959247 22178 heartbeater.cc:503] Master 127.18.129.62:46375 was elected leader, sending a full tablet report...
W20260807 15:06:46.001511 21958 tablet.cc:2369] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:06:46.098506 21750 tablet.cc:2369] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:06:46.160948 22179 tablet.cc:2369] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:46.170743 22219 raft_consensus.cc:482] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:46.170894 22219 raft_consensus.cc:504] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:46.171269 22219 leader_election.cc:288] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:46.174048 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.171623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:35181 (local address 127.18.129.3:42711)
0807 15:06:46.171675 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.171677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.171685 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:46.171712 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.171713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.171720 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.171769 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:46.171919 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.172389 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.173375 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.173728 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.173781 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.173788 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.173937 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.173974 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:06:46.174059 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.171528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:51039 (local address 127.18.129.2:40623)
0807 15:06:46.171591 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.171593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.171620 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:06:46.171700 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.171701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.171711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.171757 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:46.171906 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.172380 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.173390 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.173762 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.173829 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.173838 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.173961 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.173997 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:06:46.174069 22224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.171424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:42711 (local address 127.18.129.1:35181)
0807 15:06:46.171558 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:46.171571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:46.171627 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:46.171784 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:46.171785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:46.171788 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:46.171856 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.171858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.172464 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.172465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.173293 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.173295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.173803 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.173804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.173874 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.173879 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:06:46.173919 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:06:46.173959 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":25,"thread_start_us":59,"threads_started":1}
I20260807 15:06:46.174075 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.171436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:40623 (local address 127.18.129.1:51039)
0807 15:06:46.171564 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:46.171574 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:46.171638 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:46.171770 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:46.171770 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:46.171772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:46.171846 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.171848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.172453 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.172454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.173301 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.173303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.173839 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.173840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.173893 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.173897 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:46.173948 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:06:46.173985 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":56,"thread_start_us":70,"threads_started":1}
I20260807 15:06:46.174420 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8e60a8d38b4402a62fb781261a0bfe" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" is_pre_election: true
I20260807 15:06:46.174595 22102 raft_consensus.cc:2454] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 0.
I20260807 15:06:46.174752 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8e60a8d38b4402a62fb781261a0bfe" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636960a0dafc4904a58bb65895f443b7" is_pre_election: true
I20260807 15:06:46.174827 21573 leader_election.cc:302] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 781c2914a85f45ffa85c6f4ed1ae7faf, fa18c579e45c4a9588bf4427b78f1156; no voters:
I20260807 15:06:46.174906 21890 raft_consensus.cc:2454] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 0.
I20260807 15:06:46.174973 22219 raft_consensus.cc:2781] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:46.175033 22219 raft_consensus.cc:482] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:46.175065 22219 raft_consensus.cc:3037] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.175753 22219 raft_consensus.cc:504] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:46.175884 22219 leader_election.cc:288] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 election: Requested vote from peers 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:46.176015 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8e60a8d38b4402a62fb781261a0bfe" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636960a0dafc4904a58bb65895f443b7"
I20260807 15:06:46.176048 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8e60a8d38b4402a62fb781261a0bfe" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf"
I20260807 15:06:46.176097 21890 raft_consensus.cc:3037] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.176113 22102 raft_consensus.cc:3037] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.176770 22102 raft_consensus.cc:2454] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 1.
I20260807 15:06:46.176880 21890 raft_consensus.cc:2454] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 1.
I20260807 15:06:46.176929 21573 leader_election.cc:302] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 781c2914a85f45ffa85c6f4ed1ae7faf, fa18c579e45c4a9588bf4427b78f1156; no voters:
I20260807 15:06:46.177019 22219 raft_consensus.cc:2781] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:46.177167 22219 raft_consensus.cc:686] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 1 LEADER]: Becoming Leader. State: Replica: fa18c579e45c4a9588bf4427b78f1156, State: Running, Role: LEADER
I20260807 15:06:46.177263 22219 consensus_queue.cc:237] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } }
I20260807 15:06:46.178108 21437 catalog_manager.cc:5495] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1). New cstate: current_term: 1 leader_uuid: "fa18c579e45c4a9588bf4427b78f1156" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } health_report { overall_health: HEALTHY } } }
Aug 07 15:06:46 dist-test-slave-5bn2 krb5kdc[21371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115205, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:46.245905 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.239622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46375 (local address 127.0.0.1:51382)
0807 15:06:46.239776 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:46.239787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:46.239851 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:46.239965 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:46.239965 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:46.240289 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:46.240340 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.240342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.240931 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.240931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.241454 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.241455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.241718 (+ 263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.241718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.241755 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.242206 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:46.242210 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:46.243592 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:46.245049 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:46.245056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:46.245058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:46.245262 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:46.245413 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:46.245413 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:46.245414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:46.245462 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:46.245627 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:46.245631 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:46.245671 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 15:06:46.245717 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:06:46.245800 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260807 15:06:46.245918 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.239687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.18.129.62:46375)
0807 15:06:46.239750 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.239752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.239844 (+ 92us) server_negotiation.cc:407] Connection header received
0807 15:06:46.239904 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.239904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.239914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.239951 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:46.240396 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.240878 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.241505 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.241674 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.241714 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.243684 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:46.243685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:46.243686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:46.243692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:46.244979 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:46.245323 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:46.245323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:46.245324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:46.245369 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:46.245499 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:46.245500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:46.245500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:46.245588 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:46.245631 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.245738 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.245843 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":34}
I20260807 15:06:46.247764 21437 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:51382:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:06:46.247989 21437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:46.249446 21654 tablet_service.cc:1467] Processing CreateTablet for tablet b89cc9f6950542a28a0fdbb9d0e23d18 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=208591bab1894a83ba3fef56acecb4e8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:06:46.249464 22074 tablet_service.cc:1467] Processing CreateTablet for tablet b89cc9f6950542a28a0fdbb9d0e23d18 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=208591bab1894a83ba3fef56acecb4e8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:06:46.249540 21867 tablet_service.cc:1467] Processing CreateTablet for tablet b89cc9f6950542a28a0fdbb9d0e23d18 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=208591bab1894a83ba3fef56acecb4e8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:06:46.249619 21654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b89cc9f6950542a28a0fdbb9d0e23d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:46.249651 22074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b89cc9f6950542a28a0fdbb9d0e23d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:46.249691 21867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b89cc9f6950542a28a0fdbb9d0e23d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:46.251145 22215 tablet_bootstrap.cc:492] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: Bootstrap starting.
I20260807 15:06:46.251144 22213 tablet_bootstrap.cc:492] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: Bootstrap starting.
I20260807 15:06:46.251179 22214 tablet_bootstrap.cc:492] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: Bootstrap starting.
I20260807 15:06:46.251521 22214 tablet_bootstrap.cc:654] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:46.251797 22213 tablet_bootstrap.cc:654] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:46.251834 22215 tablet_bootstrap.cc:654] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:46.252398 22214 tablet_bootstrap.cc:492] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: No bootstrap required, opened a new log
I20260807 15:06:46.252457 22214 ts_tablet_manager.cc:1397] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:46.252513 22213 tablet_bootstrap.cc:492] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: No bootstrap required, opened a new log
I20260807 15:06:46.252563 22213 ts_tablet_manager.cc:1397] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:46.252609 22214 raft_consensus.cc:348] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.252673 22214 raft_consensus.cc:374] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:46.252691 22214 raft_consensus.cc:729] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa18c579e45c4a9588bf4427b78f1156, State: Initialized, Role: FOLLOWER
I20260807 15:06:46.252681 22213 raft_consensus.cc:348] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.252753 22213 raft_consensus.cc:374] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:46.252735 22214 consensus_queue.cc:260] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.252769 22213 raft_consensus.cc:729] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781c2914a85f45ffa85c6f4ed1ae7faf, State: Initialized, Role: FOLLOWER
I20260807 15:06:46.252805 22213 consensus_queue.cc:260] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.252820 22214 ts_tablet_manager.cc:1428] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:46.252889 22213 ts_tablet_manager.cc:1428] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:46.252889 22219 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.252997 22219 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:46.252949 22221 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.253046 22221 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:46.253530 22215 tablet_bootstrap.cc:492] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: No bootstrap required, opened a new log
I20260807 15:06:46.253578 22215 ts_tablet_manager.cc:1397] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:06:46.253707 22215 raft_consensus.cc:348] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.253765 22215 raft_consensus.cc:374] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:46.253777 22215 raft_consensus.cc:729] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 636960a0dafc4904a58bb65895f443b7, State: Initialized, Role: FOLLOWER
I20260807 15:06:46.253811 22215 consensus_queue.cc:260] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.253897 22215 ts_tablet_manager.cc:1428] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:46.253947 22220 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.254041 22220 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:46.264792 22220 raft_consensus.cc:482] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:46.264871 22220 raft_consensus.cc:504] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.265196 22220 leader_election.cc:288] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:46.268035 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.265401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:42449 (local address 127.18.129.3:42711)
0807 15:06:46.265477 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.265479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.265622 (+ 143us) server_negotiation.cc:407] Connection header received
0807 15:06:46.265655 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.265656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.265665 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.265724 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:46.265841 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.266288 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.267359 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.267717 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.267777 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.267787 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.267914 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.267950 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:06:46.268047 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.265455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:55527 (local address 127.18.129.1:35385)
0807 15:06:46.265518 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.265520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.265530 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:46.265565 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.265566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.265575 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.265631 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:46.265775 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.266272 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.267376 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.267728 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.267791 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.267799 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.267938 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.267973 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:06:46.268059 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.265339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35385 (local address 127.18.129.2:55527)
0807 15:06:46.265457 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:46.265468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:46.265524 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:46.265652 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:46.265653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:46.265656 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:46.265714 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.265716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.266345 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.266346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.267299 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.267301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.267800 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.267801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.267874 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.267877 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:46.267927 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:06:46.267965 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":39,"thread_start_us":61,"threads_started":1}
I20260807 15:06:46.268061 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.265339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:42711 (local address 127.18.129.2:42449)
0807 15:06:46.265522 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:46.265553 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:46.265613 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:46.265727 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:46.265728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:46.265729 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:46.265781 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.265783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.266358 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.266359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.267287 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.267290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.267789 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.267789 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.267853 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.267857 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:46.267900 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:06:46.267938 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":15,"thread_start_us":63,"threads_started":1}
I20260807 15:06:46.268337 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b89cc9f6950542a28a0fdbb9d0e23d18" candidate_uuid: "636960a0dafc4904a58bb65895f443b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" is_pre_election: true
I20260807 15:06:46.268436 22102 raft_consensus.cc:2454] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 636960a0dafc4904a58bb65895f443b7 in term 0.
I20260807 15:06:46.268534 21688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b89cc9f6950542a28a0fdbb9d0e23d18" candidate_uuid: "636960a0dafc4904a58bb65895f443b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa18c579e45c4a9588bf4427b78f1156" is_pre_election: true
I20260807 15:06:46.268620 21802 leader_election.cc:302] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 636960a0dafc4904a58bb65895f443b7, 781c2914a85f45ffa85c6f4ed1ae7faf; no voters:
I20260807 15:06:46.268657 21688 raft_consensus.cc:2454] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 636960a0dafc4904a58bb65895f443b7 in term 0.
I20260807 15:06:46.268818 22220 raft_consensus.cc:2781] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:46.268858 22220 raft_consensus.cc:482] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:46.268872 22220 raft_consensus.cc:3037] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.269528 22220 raft_consensus.cc:504] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.269668 22220 leader_election.cc:288] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [CANDIDATE]: Term 1 election: Requested vote from peers fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:46.269779 21688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b89cc9f6950542a28a0fdbb9d0e23d18" candidate_uuid: "636960a0dafc4904a58bb65895f443b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa18c579e45c4a9588bf4427b78f1156"
I20260807 15:06:46.269860 21688 raft_consensus.cc:3037] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.269863 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b89cc9f6950542a28a0fdbb9d0e23d18" candidate_uuid: "636960a0dafc4904a58bb65895f443b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf"
I20260807 15:06:46.269950 22102 raft_consensus.cc:3037] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.270700 21688 raft_consensus.cc:2454] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 636960a0dafc4904a58bb65895f443b7 in term 1.
I20260807 15:06:46.270700 22102 raft_consensus.cc:2454] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 636960a0dafc4904a58bb65895f443b7 in term 1.
I20260807 15:06:46.270892 21799 leader_election.cc:302] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 636960a0dafc4904a58bb65895f443b7, fa18c579e45c4a9588bf4427b78f1156; no voters:
I20260807 15:06:46.270994 22220 raft_consensus.cc:2781] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:46.271107 22220 raft_consensus.cc:686] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [term 1 LEADER]: Becoming Leader. State: Replica: 636960a0dafc4904a58bb65895f443b7, State: Running, Role: LEADER
I20260807 15:06:46.271186 22220 consensus_queue.cc:237] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.271642 22245 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: TxnStatusTablet state changed. Reason: New leader 636960a0dafc4904a58bb65895f443b7. Latest consensus state: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.271735 22245 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:06:46.271917 22248 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:06:46.271909 21435 catalog_manager.cc:5495] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 636960a0dafc4904a58bb65895f443b7 (127.18.129.2). New cstate: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } health_report { overall_health: UNKNOWN } } }
I20260807 15:06:46.272070 22248 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:06:46.283320 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.280578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:40623 (local address 127.0.0.1:38044)
0807 15:06:46.280633 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:46.280645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:46.280708 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:46.280858 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:46.280859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:46.280860 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:06:46.280917 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.280919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.281822 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.281822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.282375 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.282376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.282821 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.282822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.282909 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.282925 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:06:46.283192 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:06:46.283196 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:46.283220 (+ 24us) client_negotiation.cc:240] Negotiation successful
0807 15:06:46.283257 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 15:06:46.283329 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.280603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.18.129.2:40623)
0807 15:06:46.280658 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.280660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.280688 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:06:46.280766 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.280767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.280784 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.280858 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:46.280953 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.281756 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.282446 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.282757 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.282822 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.283005 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:46.283131 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:46.283209 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.283226 (+ 17us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.283249 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:06:46.285037 22102 raft_consensus.cc:1261] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Refusing update from remote peer 636960a0dafc4904a58bb65895f443b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:46.285352 21688 raft_consensus.cc:1261] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Refusing update from remote peer 636960a0dafc4904a58bb65895f443b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:46.285658 22220 consensus_queue.cc:1035] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:46.287149 22221 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: TxnStatusTablet state changed. Reason: New leader 636960a0dafc4904a58bb65895f443b7. Latest consensus state: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.287253 22221 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:46.287467 22220 consensus_queue.cc:1035] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:46.288491 22245 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.288580 22245 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:06:46.288769 22219 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: TxnStatusTablet state changed. Reason: New leader 636960a0dafc4904a58bb65895f443b7. Latest consensus state: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.288853 22219 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:46.289053 22221 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.289134 22221 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P 781c2914a85f45ffa85c6f4ed1ae7faf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:46.289680 22253 mvcc.cc:205] Tried to move back new op lower bound from 7315927884941086720 to 7315927884887355392. Current Snapshot: MvccSnapshot[applied={T|T < 7315927884941086720 or (T in {7315927884941086720})}]
I20260807 15:06:46.290283 22245 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.290360 22245 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P 636960a0dafc4904a58bb65895f443b7: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:06:46.291579 22219 tablet_replica.cc:439] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } } }
I20260807 15:06:46.291666 22219 tablet_replica.cc:442] T b89cc9f6950542a28a0fdbb9d0e23d18 P fa18c579e45c4a9588bf4427b78f1156: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:06:46.293968 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.291243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.18.129.1:35385)
0807 15:06:46.291277 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.291279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.291286 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:06:46.291413 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.291413 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.291432 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.291486 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:46.291689 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.292424 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.293169 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.293475 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.293550 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.293664 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:46.293783 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:46.293855 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.293879 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.293899 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:06:46.298916 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.296340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:37271 (local address 127.18.129.2:40623)
0807 15:06:46.296383 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.296385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.296568 (+ 183us) server_negotiation.cc:407] Connection header received
0807 15:06:46.296598 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.296599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.296611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.296663 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:46.296754 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.297445 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.298177 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.298478 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.298555 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.298685 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:46.298718 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:46.298759 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.298799 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.298818 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:46.298951 22272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.296280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:40623 (local address 127.18.129.1:37271)
0807 15:06:46.296467 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:46.296477 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:46.296531 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:46.296666 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:46.296667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:46.296668 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:06:46.296716 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.296717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.297515 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.297516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.298109 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.298111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.298550 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.298550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.298600 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.298613 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:06:46.298768 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:06:46.298769 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:06:46.298794 (+ 25us) client_negotiation.cc:240] Negotiation successful
0807 15:06:46.298829 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260807 15:06:46.301402 22102 raft_consensus.cc:1261] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Refusing update from remote peer fa18c579e45c4a9588bf4427b78f1156: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:46.301465 21890 raft_consensus.cc:1261] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Refusing update from remote peer fa18c579e45c4a9588bf4427b78f1156: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:46.301673 22226 consensus_queue.cc:1035] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Connected to new peer: Peer: permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:46.301805 22219 consensus_queue.cc:1035] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Connected to new peer: Peer: permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:46.303519 22252 mvcc.cc:205] Tried to move back new op lower bound from 7315927885008273408 to 7315927884502757376. Current Snapshot: MvccSnapshot[applied={T|T < 7315927885008273408}]
I20260807 15:06:46.310757 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.307805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35385 (local address 127.18.129.2:53123)
0807 15:06:46.308043 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:46.308054 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:46.308159 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:46.308270 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:46.308271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:46.308272 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:06:46.308332 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.308333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.309164 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.309164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.309966 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:46.309969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.310425 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.310426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:46.310468 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.310482 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:06:46.310623 (+ 141us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:06:46.310625 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:06:46.310653 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 15:06:46.310684 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":159,"threads_started":1}
I20260807 15:06:46.310751 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:46.307836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:53123 (local address 127.18.129.1:35385)
0807 15:06:46.307872 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:46.307873 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:46.308166 (+ 293us) server_negotiation.cc:407] Connection header received
0807 15:06:46.308193 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:46.308194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:46.308204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:46.308268 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:46.308367 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.309098 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.310040 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:46.310349 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:46.310417 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:46.310546 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:46.310579 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:46.310623 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:46.310657 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:06:46.310677 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:06:46.320099 21435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:46.320215 21435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:46.320976 21435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: Sending DeleteTablet for 3 replicas of tablet 8e8e60a8d38b4402a62fb781261a0bfe
I20260807 15:06:46.321399 21867 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e8e60a8d38b4402a62fb781261a0bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:46 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:38032
I20260807 15:06:46.321401 22074 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e8e60a8d38b4402a62fb781261a0bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:46 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:53932
I20260807 15:06:46.321799 21654 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e8e60a8d38b4402a62fb781261a0bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:46 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41908
I20260807 15:06:46.322109 22287 tablet_replica.cc:330] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: stopping tablet replica
I20260807 15:06:46.322184 21435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:06:46.322257 22287 raft_consensus.cc:2229] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 1 LEADER]: Raft consensus shutting down.
W20260807 15:06:46.322336 21435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:46.322546 22287 raft_consensus.cc:2258] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:46.323084 22287 ts_tablet_manager.cc:1905] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:46.323563 21867 tablet_service.cc:1467] Processing CreateTablet for tablet 4b12d8ccab2e41bb9725cc57baf68d10 (DEFAULT_TABLE table=test-table [id=6e93a9746d2847009dc08f9260635616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:46.323696 21867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b12d8ccab2e41bb9725cc57baf68d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:46.325059 22215 tablet_bootstrap.cc:492] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: Bootstrap starting.
I20260807 15:06:46.325086 22287 ts_tablet_manager.cc:1918] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:46.325313 22287 log.cc:1196] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/8e8e60a8d38b4402a62fb781261a0bfe
I20260807 15:06:46.325482 22215 tablet_bootstrap.cc:654] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:46.325749 22287 ts_tablet_manager.cc:1939] T 8e8e60a8d38b4402a62fb781261a0bfe P fa18c579e45c4a9588bf4427b78f1156: Deleting consensus metadata
I20260807 15:06:46.326014 22074 tablet_service.cc:1467] Processing CreateTablet for tablet 4b12d8ccab2e41bb9725cc57baf68d10 (DEFAULT_TABLE table=test-table [id=6e93a9746d2847009dc08f9260635616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:46.326148 22074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b12d8ccab2e41bb9725cc57baf68d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:46.326978 21415 catalog_manager.cc:4841] TS fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385): tablet 8e8e60a8d38b4402a62fb781261a0bfe (table test-table [id=92c270720a0042d1a7d18d0ade24457a]) successfully deleted
I20260807 15:06:46.327054 21654 tablet_service.cc:1467] Processing CreateTablet for tablet 4b12d8ccab2e41bb9725cc57baf68d10 (DEFAULT_TABLE table=test-table [id=6e93a9746d2847009dc08f9260635616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:46.327150 21654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b12d8ccab2e41bb9725cc57baf68d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:46.327538 22213 tablet_bootstrap.cc:492] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: Bootstrap starting.
I20260807 15:06:46.327709 22286 tablet_replica.cc:330] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: stopping tablet replica
I20260807 15:06:46.327827 22286 raft_consensus.cc:2229] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:46.327884 22286 raft_consensus.cc:2258] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:46.327944 22213 tablet_bootstrap.cc:654] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:46.328332 22286 ts_tablet_manager.cc:1905] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:46.328433 22214 tablet_bootstrap.cc:492] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: Bootstrap starting.
I20260807 15:06:46.328837 22214 tablet_bootstrap.cc:654] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:46.329480 22214 tablet_bootstrap.cc:492] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: No bootstrap required, opened a new log
I20260807 15:06:46.329527 22214 ts_tablet_manager.cc:1397] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:46.329702 22214 raft_consensus.cc:348] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.329777 22214 raft_consensus.cc:374] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:46.329795 22214 raft_consensus.cc:729] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa18c579e45c4a9588bf4427b78f1156, State: Initialized, Role: FOLLOWER
I20260807 15:06:46.329835 22214 consensus_queue.cc:260] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.329923 22214 ts_tablet_manager.cc:1428] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:46.330067 22286 ts_tablet_manager.cc:1918] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:46.330188 22286 log.cc:1196] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/8e8e60a8d38b4402a62fb781261a0bfe
I20260807 15:06:46.330619 22286 ts_tablet_manager.cc:1939] T 8e8e60a8d38b4402a62fb781261a0bfe P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting consensus metadata
I20260807 15:06:46.331064 21417 catalog_manager.cc:4841] TS 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711): tablet 8e8e60a8d38b4402a62fb781261a0bfe (table test-table [id=92c270720a0042d1a7d18d0ade24457a]) successfully deleted
I20260807 15:06:46.331535 22213 tablet_bootstrap.cc:492] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: No bootstrap required, opened a new log
I20260807 15:06:46.331593 22213 ts_tablet_manager.cc:1397] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 15:06:46.331725 22213 raft_consensus.cc:348] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.331784 22213 raft_consensus.cc:374] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:46.331801 22213 raft_consensus.cc:729] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781c2914a85f45ffa85c6f4ed1ae7faf, State: Initialized, Role: FOLLOWER
I20260807 15:06:46.331861 22213 consensus_queue.cc:260] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.331938 22213 ts_tablet_manager.cc:1428] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:46.335021 22283 tablet_replica.cc:330] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: stopping tablet replica
I20260807 15:06:46.335110 22215 tablet_bootstrap.cc:492] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: No bootstrap required, opened a new log
I20260807 15:06:46.335172 22215 ts_tablet_manager.cc:1397] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20260807 15:06:46.335314 22283 raft_consensus.cc:2229] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:46.335421 22283 raft_consensus.cc:2258] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:46.335321 22215 raft_consensus.cc:348] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.335799 22215 raft_consensus.cc:374] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:46.335829 22215 raft_consensus.cc:729] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 636960a0dafc4904a58bb65895f443b7, State: Initialized, Role: FOLLOWER
I20260807 15:06:46.335870 22215 consensus_queue.cc:260] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.335966 22215 ts_tablet_manager.cc:1428] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:06:46.336025 22283 ts_tablet_manager.cc:1905] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:46.338147 22283 ts_tablet_manager.cc:1918] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:46.338403 22283 log.cc:1196] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/8e8e60a8d38b4402a62fb781261a0bfe
I20260807 15:06:46.338748 22283 ts_tablet_manager.cc:1939] T 8e8e60a8d38b4402a62fb781261a0bfe P 636960a0dafc4904a58bb65895f443b7: Deleting consensus metadata
I20260807 15:06:46.339231 21415 catalog_manager.cc:4841] TS 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623): tablet 8e8e60a8d38b4402a62fb781261a0bfe (table test-table [id=92c270720a0042d1a7d18d0ade24457a]) successfully deleted
I20260807 15:06:46.418490 22226 raft_consensus.cc:482] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:46.418597 22226 raft_consensus.cc:504] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.418771 22226 leader_election.cc:288] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:46.419004 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b12d8ccab2e41bb9725cc57baf68d10" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636960a0dafc4904a58bb65895f443b7" is_pre_election: true
I20260807 15:06:46.418996 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b12d8ccab2e41bb9725cc57baf68d10" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" is_pre_election: true
I20260807 15:06:46.419099 21890 raft_consensus.cc:2454] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 0.
I20260807 15:06:46.419099 22102 raft_consensus.cc:2454] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 0.
I20260807 15:06:46.419281 21570 leader_election.cc:302] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 636960a0dafc4904a58bb65895f443b7, fa18c579e45c4a9588bf4427b78f1156; no voters:
I20260807 15:06:46.419406 22226 raft_consensus.cc:2781] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:46.419445 22226 raft_consensus.cc:482] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:46.419463 22226 raft_consensus.cc:3037] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.420168 22226 raft_consensus.cc:504] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.420302 22226 leader_election.cc:288] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 election: Requested vote from peers 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:46.420455 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b12d8ccab2e41bb9725cc57baf68d10" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf"
I20260807 15:06:46.420475 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b12d8ccab2e41bb9725cc57baf68d10" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636960a0dafc4904a58bb65895f443b7"
I20260807 15:06:46.420548 22102 raft_consensus.cc:3037] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.420584 21890 raft_consensus.cc:3037] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:46.421257 22102 raft_consensus.cc:2454] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 1.
I20260807 15:06:46.421330 21890 raft_consensus.cc:2454] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 1.
I20260807 15:06:46.421427 21573 leader_election.cc:302] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 781c2914a85f45ffa85c6f4ed1ae7faf, fa18c579e45c4a9588bf4427b78f1156; no voters:
I20260807 15:06:46.421532 22226 raft_consensus.cc:2781] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:46.421602 22226 raft_consensus.cc:686] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 1 LEADER]: Becoming Leader. State: Replica: fa18c579e45c4a9588bf4427b78f1156, State: Running, Role: LEADER
I20260807 15:06:46.421654 22226 consensus_queue.cc:237] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:46.422259 21435 catalog_manager.cc:5495] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1). New cstate: current_term: 1 leader_uuid: "fa18c579e45c4a9588bf4427b78f1156" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } health_report { overall_health: UNKNOWN } } }
I20260807 15:06:46.450125 21890 raft_consensus.cc:1261] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Refusing update from remote peer fa18c579e45c4a9588bf4427b78f1156: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:46.450130 22102 raft_consensus.cc:1261] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Refusing update from remote peer fa18c579e45c4a9588bf4427b78f1156: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:46.450366 22226 consensus_queue.cc:1035] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Connected to new peer: Peer: permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:46.450493 22219 consensus_queue.cc:1035] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Connected to new peer: Peer: permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:46.451412 22257 mvcc.cc:205] Tried to move back new op lower bound from 7315927885618180096 to 7315927885503430656. Current Snapshot: MvccSnapshot[applied={T|T < 7315927885618180096}]
W20260807 15:06:46.455017 22257 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:06:47.972595 21435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:47.972699 21435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:47.973435 21435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: Sending DeleteTablet for 3 replicas of tablet 4b12d8ccab2e41bb9725cc57baf68d10
I20260807 15:06:47.973692 21654 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b12d8ccab2e41bb9725cc57baf68d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:47 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41908
I20260807 15:06:47.973733 21866 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b12d8ccab2e41bb9725cc57baf68d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:47 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:38032
I20260807 15:06:47.974006 22074 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b12d8ccab2e41bb9725cc57baf68d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:47 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:53932
I20260807 15:06:47.974236 22333 tablet_replica.cc:330] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: stopping tablet replica
I20260807 15:06:47.974301 22333 raft_consensus.cc:2229] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:47.974344 22333 raft_consensus.cc:2258] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:47.974561 22333 ts_tablet_manager.cc:1905] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:47.976563 22331 tablet_replica.cc:330] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: stopping tablet replica
I20260807 15:06:47.976629 22331 raft_consensus.cc:2229] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:06:47.976686 22331 raft_consensus.cc:2258] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:47.976790 22332 tablet_replica.cc:330] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: stopping tablet replica
I20260807 15:06:47.976846 22332 raft_consensus.cc:2229] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:47.976882 22332 raft_consensus.cc:2258] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:47.977103 22332 ts_tablet_manager.cc:1905] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:47.977133 22331 ts_tablet_manager.cc:1905] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:47.978112 22333 ts_tablet_manager.cc:1918] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:06:47.978235 22333 log.cc:1196] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/4b12d8ccab2e41bb9725cc57baf68d10
I20260807 15:06:47.978564 22333 ts_tablet_manager.cc:1939] T 4b12d8ccab2e41bb9725cc57baf68d10 P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting consensus metadata
I20260807 15:06:47.978708 22332 ts_tablet_manager.cc:1918] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:06:47.978708 22331 ts_tablet_manager.cc:1918] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:06:47.978768 22331 log.cc:1196] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/4b12d8ccab2e41bb9725cc57baf68d10
I20260807 15:06:47.978873 22332 log.cc:1196] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/4b12d8ccab2e41bb9725cc57baf68d10
I20260807 15:06:47.978956 21417 catalog_manager.cc:4841] TS 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711): tablet 4b12d8ccab2e41bb9725cc57baf68d10 (table test-table [id=6e93a9746d2847009dc08f9260635616]) successfully deleted
I20260807 15:06:47.979058 22331 ts_tablet_manager.cc:1939] T 4b12d8ccab2e41bb9725cc57baf68d10 P fa18c579e45c4a9588bf4427b78f1156: Deleting consensus metadata
I20260807 15:06:47.979164 22332 ts_tablet_manager.cc:1939] T 4b12d8ccab2e41bb9725cc57baf68d10 P 636960a0dafc4904a58bb65895f443b7: Deleting consensus metadata
I20260807 15:06:47.979386 21415 catalog_manager.cc:4841] TS fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385): tablet 4b12d8ccab2e41bb9725cc57baf68d10 (table test-table [id=6e93a9746d2847009dc08f9260635616]) successfully deleted
I20260807 15:06:47.979470 21415 catalog_manager.cc:4841] TS 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623): tablet 4b12d8ccab2e41bb9725cc57baf68d10 (table test-table [id=6e93a9746d2847009dc08f9260635616]) successfully deleted
Aug 07 15:06:47 dist-test-slave-5bn2 krb5kdc[21371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115207, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:06:47 dist-test-slave-5bn2 krb5kdc[21371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115207, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:48.002286 22353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:47.990726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.18.129.62:46375)
0807 15:06:47.990882 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:47.990884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:47.990896 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:06:47.990946 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:47.990946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:47.990957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:47.991015 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:47.991548 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:47.992109 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:47.992845 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:47.993065 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:47.993127 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:47.999926 (+ 6799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:47.999927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:47.999927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:47.999934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:48.001391 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:48.001711 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:48.001711 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:48.001712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:48.001766 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:48.001898 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:48.001898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:48.001899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:48.002002 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:48.002073 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:48.002144 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:06:48.002206 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260807 15:06:48.003717 21435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:48.003875 21435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:48.005139 21654 tablet_service.cc:1467] Processing CreateTablet for tablet ba69abf92328489eb9039fb2bef712f2 (DEFAULT_TABLE table=test-table [id=b3ba8becdb2d4fa2b3b7502ed2952d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:48.005153 22074 tablet_service.cc:1467] Processing CreateTablet for tablet ba69abf92328489eb9039fb2bef712f2 (DEFAULT_TABLE table=test-table [id=b3ba8becdb2d4fa2b3b7502ed2952d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:48.005180 21866 tablet_service.cc:1467] Processing CreateTablet for tablet ba69abf92328489eb9039fb2bef712f2 (DEFAULT_TABLE table=test-table [id=b3ba8becdb2d4fa2b3b7502ed2952d4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:48.005286 22074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba69abf92328489eb9039fb2bef712f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:48.005287 21654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba69abf92328489eb9039fb2bef712f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:48.005295 21866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba69abf92328489eb9039fb2bef712f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:48.006321 22354 tablet_bootstrap.cc:492] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: Bootstrap starting.
I20260807 15:06:48.006708 22354 tablet_bootstrap.cc:654] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:48.006735 22356 tablet_bootstrap.cc:492] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: Bootstrap starting.
I20260807 15:06:48.006979 22355 tablet_bootstrap.cc:492] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: Bootstrap starting.
I20260807 15:06:48.007165 22356 tablet_bootstrap.cc:654] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:48.007347 22355 tablet_bootstrap.cc:654] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:48.007515 22354 tablet_bootstrap.cc:492] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: No bootstrap required, opened a new log
I20260807 15:06:48.007575 22354 ts_tablet_manager.cc:1397] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:48.007809 22356 tablet_bootstrap.cc:492] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: No bootstrap required, opened a new log
I20260807 15:06:48.007764 22354 raft_consensus.cc:348] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.007844 22354 raft_consensus.cc:374] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:48.007866 22354 raft_consensus.cc:729] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781c2914a85f45ffa85c6f4ed1ae7faf, State: Initialized, Role: FOLLOWER
I20260807 15:06:48.007858 22356 ts_tablet_manager.cc:1397] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:48.007913 22354 consensus_queue.cc:260] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.007987 22356 raft_consensus.cc:348] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.008064 22356 raft_consensus.cc:374] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:48.008085 22354 ts_tablet_manager.cc:1428] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:48.008101 22355 tablet_bootstrap.cc:492] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: No bootstrap required, opened a new log
I20260807 15:06:48.008119 22356 raft_consensus.cc:729] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 636960a0dafc4904a58bb65895f443b7, State: Initialized, Role: FOLLOWER
I20260807 15:06:48.008143 22355 ts_tablet_manager.cc:1397] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:48.008190 22356 consensus_queue.cc:260] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.008299 22355 raft_consensus.cc:348] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.008383 22355 raft_consensus.cc:374] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:48.008406 22355 raft_consensus.cc:729] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa18c579e45c4a9588bf4427b78f1156, State: Initialized, Role: FOLLOWER
I20260807 15:06:48.008345 22356 ts_tablet_manager.cc:1428] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:48.008450 22355 consensus_queue.cc:260] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.008538 22355 ts_tablet_manager.cc:1428] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:48.149937 22310 raft_consensus.cc:482] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:48.150065 22310 raft_consensus.cc:504] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.150265 22310 leader_election.cc:288] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:48.150436 21688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba69abf92328489eb9039fb2bef712f2" candidate_uuid: "636960a0dafc4904a58bb65895f443b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa18c579e45c4a9588bf4427b78f1156" is_pre_election: true
I20260807 15:06:48.150456 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba69abf92328489eb9039fb2bef712f2" candidate_uuid: "636960a0dafc4904a58bb65895f443b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" is_pre_election: true
I20260807 15:06:48.150534 21688 raft_consensus.cc:2454] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 636960a0dafc4904a58bb65895f443b7 in term 0.
I20260807 15:06:48.150558 22102 raft_consensus.cc:2454] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 636960a0dafc4904a58bb65895f443b7 in term 0.
I20260807 15:06:48.150689 21799 leader_election.cc:302] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 636960a0dafc4904a58bb65895f443b7, fa18c579e45c4a9588bf4427b78f1156; no voters:
I20260807 15:06:48.150810 22310 raft_consensus.cc:2781] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:48.150843 22310 raft_consensus.cc:482] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:48.150884 22310 raft_consensus.cc:3037] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:48.151561 22310 raft_consensus.cc:504] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.151700 22310 leader_election.cc:288] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [CANDIDATE]: Term 1 election: Requested vote from peers fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:48.151871 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba69abf92328489eb9039fb2bef712f2" candidate_uuid: "636960a0dafc4904a58bb65895f443b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf"
I20260807 15:06:48.151940 22102 raft_consensus.cc:3037] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:48.151892 21688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba69abf92328489eb9039fb2bef712f2" candidate_uuid: "636960a0dafc4904a58bb65895f443b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa18c579e45c4a9588bf4427b78f1156"
I20260807 15:06:48.151968 21688 raft_consensus.cc:3037] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:48.152540 22102 raft_consensus.cc:2454] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 636960a0dafc4904a58bb65895f443b7 in term 1.
I20260807 15:06:48.152583 21688 raft_consensus.cc:2454] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 636960a0dafc4904a58bb65895f443b7 in term 1.
I20260807 15:06:48.152709 21802 leader_election.cc:302] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 636960a0dafc4904a58bb65895f443b7, 781c2914a85f45ffa85c6f4ed1ae7faf; no voters:
I20260807 15:06:48.152818 22310 raft_consensus.cc:2781] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:48.152881 22310 raft_consensus.cc:686] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 1 LEADER]: Becoming Leader. State: Replica: 636960a0dafc4904a58bb65895f443b7, State: Running, Role: LEADER
I20260807 15:06:48.152933 22310 consensus_queue.cc:237] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.153568 21435 catalog_manager.cc:5495] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 636960a0dafc4904a58bb65895f443b7 (127.18.129.2). New cstate: current_term: 1 leader_uuid: "636960a0dafc4904a58bb65895f443b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } health_report { overall_health: UNKNOWN } } }
I20260807 15:06:48.192021 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:48.189830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.18.129.2:40623)
0807 15:06:48.190008 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:48.190010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:48.190018 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:48.190039 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:48.190040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:48.190049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:48.190116 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:48.190233 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:48.190790 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:48.191519 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:48.191715 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:48.191764 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:48.191855 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:48.191880 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:48.191910 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:48.191942 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 15:06:48.191957 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260807 15:06:48.196022 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:48.194018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:53423 (local address 127.18.129.2:40623)
0807 15:06:48.194072 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:48.194073 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:48.194139 (+ 66us) server_negotiation.cc:407] Connection header received
0807 15:06:48.194186 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:48.194187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:48.194194 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:48.194237 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:48.194312 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:48.194730 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:48.195520 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:48.195696 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:48.195741 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:48.195830 (+ 89us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:48.195851 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:48.195881 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:48.195934 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 15:06:48.195949 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:06:48.196034 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:48.193964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:40623 (local address 127.18.129.2:53423)
0807 15:06:48.194092 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:48.194101 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:48.194151 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:48.194245 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:48.194246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:48.194247 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:06:48.194279 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:48.194280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:48.194776 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:48.194776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:48.195447 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:48.195450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:48.195746 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:48.195746 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:48.195782 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:48.195785 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:06:48.195885 (+ 100us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:06:48.195887 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:06:48.195933 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:06:48.195965 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260807 15:06:48.197933 22102 raft_consensus.cc:1261] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Refusing update from remote peer 636960a0dafc4904a58bb65895f443b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:48.197937 21688 raft_consensus.cc:1261] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Refusing update from remote peer 636960a0dafc4904a58bb65895f443b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:48.198159 22260 consensus_queue.cc:1035] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:48.198336 22260 consensus_queue.cc:1035] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:48.208261 21437 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:48.208372 21437 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:48.208956 21437 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: Sending DeleteTablet for 3 replicas of tablet ba69abf92328489eb9039fb2bef712f2
I20260807 15:06:48.209188 22074 tablet_service.cc:1514] Processing DeleteTablet for tablet ba69abf92328489eb9039fb2bef712f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:48 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:53932
I20260807 15:06:48.209215 21866 tablet_service.cc:1514] Processing DeleteTablet for tablet ba69abf92328489eb9039fb2bef712f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:48 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:38032
I20260807 15:06:48.209234 21654 tablet_service.cc:1514] Processing DeleteTablet for tablet ba69abf92328489eb9039fb2bef712f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:48 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41908
I20260807 15:06:48.209287 22332 tablet_replica.cc:330] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: stopping tablet replica
I20260807 15:06:48.209338 22332 raft_consensus.cc:2229] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:06:48.209335 22333 tablet_replica.cc:330] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: stopping tablet replica
I20260807 15:06:48.209390 22333 raft_consensus.cc:2229] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:48.209412 22332 raft_consensus.cc:2258] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:48.209429 22333 raft_consensus.cc:2258] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:48.209496 22331 tablet_replica.cc:330] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: stopping tablet replica
I20260807 15:06:48.209546 22331 raft_consensus.cc:2229] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:48.209576 22331 raft_consensus.cc:2258] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:48.209724 22332 ts_tablet_manager.cc:1905] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:48.209888 22331 ts_tablet_manager.cc:1905] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:48.211179 22332 ts_tablet_manager.cc:1918] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:48.211251 22332 log.cc:1196] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/ba69abf92328489eb9039fb2bef712f2
I20260807 15:06:48.211440 22332 ts_tablet_manager.cc:1939] T ba69abf92328489eb9039fb2bef712f2 P 636960a0dafc4904a58bb65895f443b7: Deleting consensus metadata
I20260807 15:06:48.211459 22331 ts_tablet_manager.cc:1918] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:48.211526 22331 log.cc:1196] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/ba69abf92328489eb9039fb2bef712f2
I20260807 15:06:48.211728 22331 ts_tablet_manager.cc:1939] T ba69abf92328489eb9039fb2bef712f2 P fa18c579e45c4a9588bf4427b78f1156: Deleting consensus metadata
I20260807 15:06:48.211719 21415 catalog_manager.cc:4841] TS 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623): tablet ba69abf92328489eb9039fb2bef712f2 (table test-table [id=b3ba8becdb2d4fa2b3b7502ed2952d4e]) successfully deleted
I20260807 15:06:48.212008 21415 catalog_manager.cc:4841] TS fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385): tablet ba69abf92328489eb9039fb2bef712f2 (table test-table [id=b3ba8becdb2d4fa2b3b7502ed2952d4e]) successfully deleted
I20260807 15:06:48.212285 22333 ts_tablet_manager.cc:1905] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:48.213500 22333 ts_tablet_manager.cc:1918] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:06:48.213557 22333 log.cc:1196] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/ba69abf92328489eb9039fb2bef712f2
I20260807 15:06:48.213738 22333 ts_tablet_manager.cc:1939] T ba69abf92328489eb9039fb2bef712f2 P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting consensus metadata
I20260807 15:06:48.214154 21417 catalog_manager.cc:4841] TS 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711): tablet ba69abf92328489eb9039fb2bef712f2 (table test-table [id=b3ba8becdb2d4fa2b3b7502ed2952d4e]) successfully deleted
I20260807 15:06:48.214123 21437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:48.214303 21437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:48.215515 22074 tablet_service.cc:1467] Processing CreateTablet for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a (DEFAULT_TABLE table=test-table [id=72ecddbf97e34af6bf8738aa4b8e716f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:48.215515 21654 tablet_service.cc:1467] Processing CreateTablet for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a (DEFAULT_TABLE table=test-table [id=72ecddbf97e34af6bf8738aa4b8e716f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:48.215631 22074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:48.215631 21654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:48.216776 22354 tablet_bootstrap.cc:492] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: Bootstrap starting.
I20260807 15:06:48.216928 21866 tablet_service.cc:1467] Processing CreateTablet for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a (DEFAULT_TABLE table=test-table [id=72ecddbf97e34af6bf8738aa4b8e716f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:48.217031 21866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:48.217140 22354 tablet_bootstrap.cc:654] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:48.217437 22355 tablet_bootstrap.cc:492] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156: Bootstrap starting.
I20260807 15:06:48.217767 22355 tablet_bootstrap.cc:654] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:48.217996 22356 tablet_bootstrap.cc:492] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: Bootstrap starting.
I20260807 15:06:48.218273 22356 tablet_bootstrap.cc:654] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:48.218312 22355 tablet_bootstrap.cc:492] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156: No bootstrap required, opened a new log
I20260807 15:06:48.218346 22355 ts_tablet_manager.cc:1397] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:48.218444 22355 raft_consensus.cc:348] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.218490 22355 raft_consensus.cc:374] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:48.218508 22355 raft_consensus.cc:729] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa18c579e45c4a9588bf4427b78f1156, State: Initialized, Role: FOLLOWER
I20260807 15:06:48.218556 22355 consensus_queue.cc:260] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.218647 22355 ts_tablet_manager.cc:1428] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:48.218915 22356 tablet_bootstrap.cc:492] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: No bootstrap required, opened a new log
I20260807 15:06:48.218961 22356 ts_tablet_manager.cc:1397] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:06:48.219050 22356 raft_consensus.cc:348] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.219106 22356 raft_consensus.cc:374] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:48.219125 22356 raft_consensus.cc:729] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 636960a0dafc4904a58bb65895f443b7, State: Initialized, Role: FOLLOWER
I20260807 15:06:48.219170 22356 consensus_queue.cc:260] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.219250 22356 ts_tablet_manager.cc:1428] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:48.219529 22354 tablet_bootstrap.cc:492] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: No bootstrap required, opened a new log
I20260807 15:06:48.219571 22354 ts_tablet_manager.cc:1397] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:06:48.219702 22354 raft_consensus.cc:348] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.219758 22354 raft_consensus.cc:374] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:48.219775 22354 raft_consensus.cc:729] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781c2914a85f45ffa85c6f4ed1ae7faf, State: Initialized, Role: FOLLOWER
I20260807 15:06:48.219820 22354 consensus_queue.cc:260] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.219893 22354 ts_tablet_manager.cc:1428] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:48.370132 22318 raft_consensus.cc:482] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:48.370239 22318 raft_consensus.cc:504] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.370424 22318 leader_election.cc:288] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:48.370637 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754f8a8dea5b447fb9c3e6ca63ba0c2a" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" is_pre_election: true
I20260807 15:06:48.370637 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754f8a8dea5b447fb9c3e6ca63ba0c2a" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636960a0dafc4904a58bb65895f443b7" is_pre_election: true
I20260807 15:06:48.370738 21890 raft_consensus.cc:2454] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 0.
I20260807 15:06:48.370738 22102 raft_consensus.cc:2454] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 0.
I20260807 15:06:48.370914 21570 leader_election.cc:302] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 636960a0dafc4904a58bb65895f443b7, fa18c579e45c4a9588bf4427b78f1156; no voters:
I20260807 15:06:48.371057 22318 raft_consensus.cc:2781] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:48.371117 22318 raft_consensus.cc:482] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:48.371137 22318 raft_consensus.cc:3037] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:48.371846 22318 raft_consensus.cc:504] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.371968 22318 leader_election.cc:288] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 election: Requested vote from peers 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623), 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711)
I20260807 15:06:48.372179 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754f8a8dea5b447fb9c3e6ca63ba0c2a" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf"
I20260807 15:06:48.372179 21890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "754f8a8dea5b447fb9c3e6ca63ba0c2a" candidate_uuid: "fa18c579e45c4a9588bf4427b78f1156" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "636960a0dafc4904a58bb65895f443b7"
I20260807 15:06:48.372268 22102 raft_consensus.cc:3037] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:48.372268 21890 raft_consensus.cc:3037] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:48.373057 22102 raft_consensus.cc:2454] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 1.
I20260807 15:06:48.373059 21890 raft_consensus.cc:2454] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa18c579e45c4a9588bf4427b78f1156 in term 1.
I20260807 15:06:48.373245 21573 leader_election.cc:302] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 781c2914a85f45ffa85c6f4ed1ae7faf, fa18c579e45c4a9588bf4427b78f1156; no voters:
I20260807 15:06:48.373358 22318 raft_consensus.cc:2781] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:48.373473 22318 raft_consensus.cc:686] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [term 1 LEADER]: Becoming Leader. State: Replica: fa18c579e45c4a9588bf4427b78f1156, State: Running, Role: LEADER
I20260807 15:06:48.373526 22318 consensus_queue.cc:237] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } }
I20260807 15:06:48.374115 21437 catalog_manager.cc:5495] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1). New cstate: current_term: 1 leader_uuid: "fa18c579e45c4a9588bf4427b78f1156" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa18c579e45c4a9588bf4427b78f1156" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 } health_report { overall_health: UNKNOWN } } }
I20260807 15:06:48.404983 22383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:48.402681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.18.129.1:35385)
0807 15:06:48.402836 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:48.402838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:48.402843 (+ 5us) server_negotiation.cc:407] Connection header received
0807 15:06:48.402883 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:48.402884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:48.402894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:48.402944 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:48.403065 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:48.403540 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:48.404292 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:48.404597 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:48.404667 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:48.404755 (+ 88us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:48.404790 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:48.404829 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:48.404882 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 15:06:48.404904 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260807 15:06:48.407128 21890 raft_consensus.cc:1261] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Refusing update from remote peer fa18c579e45c4a9588bf4427b78f1156: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:48.407130 22102 raft_consensus.cc:1261] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Refusing update from remote peer fa18c579e45c4a9588bf4427b78f1156: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:48.407344 22323 consensus_queue.cc:1035] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Connected to new peer: Peer: permanent_uuid: "781c2914a85f45ffa85c6f4ed1ae7faf" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:48.407469 22318 consensus_queue.cc:1035] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P fa18c579e45c4a9588bf4427b78f1156 [LEADER]: Connected to new peer: Peer: permanent_uuid: "636960a0dafc4904a58bb65895f443b7" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:06:48.411700 22304 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:06:49.928508 21435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:49.928614 21435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:49.929518 21435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6808e678c19d495b80c3d8a5b6aed517: Sending DeleteTablet for 3 replicas of tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a
I20260807 15:06:49.929843 21866 tablet_service.cc:1514] Processing DeleteTablet for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:49 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:38032
I20260807 15:06:49.929859 22074 tablet_service.cc:1514] Processing DeleteTablet for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:49 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:53932
I20260807 15:06:49.929869 21654 tablet_service.cc:1514] Processing DeleteTablet for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:49 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41908
I20260807 15:06:49.929916 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 21551
I20260807 15:06:49.934998 22419 tablet_replica.cc:330] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: stopping tablet replica
I20260807 15:06:49.935119 22419 raft_consensus.cc:2229] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:49.935191 22419 raft_consensus.cc:2258] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:49.935451 22419 ts_tablet_manager.cc:1905] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:49.937294 22419 ts_tablet_manager.cc:1918] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:06:49.937386 22419 log.cc:1196] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/754f8a8dea5b447fb9c3e6ca63ba0c2a
I20260807 15:06:49.937700 22419 ts_tablet_manager.cc:1939] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 781c2914a85f45ffa85c6f4ed1ae7faf: Deleting consensus metadata
I20260807 15:06:49.938122 21417 catalog_manager.cc:4841] TS 781c2914a85f45ffa85c6f4ed1ae7faf (127.18.129.3:42711): tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a (table test-table [id=72ecddbf97e34af6bf8738aa4b8e716f]) successfully deleted
I20260807 15:06:49.941649 22420 tablet_replica.cc:330] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: stopping tablet replica
I20260807 15:06:49.941732 22420 raft_consensus.cc:2229] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:49.941771 22420 raft_consensus.cc:2258] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:49.942047 22420 ts_tablet_manager.cc:1905] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:49.943754 22420 ts_tablet_manager.cc:1918] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:06:49.943825 22420 log.cc:1196] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/754f8a8dea5b447fb9c3e6ca63ba0c2a
I20260807 15:06:49.944013 22420 ts_tablet_manager.cc:1939] T 754f8a8dea5b447fb9c3e6ca63ba0c2a P 636960a0dafc4904a58bb65895f443b7: Deleting consensus metadata
I20260807 15:06:49.944537 21415 catalog_manager.cc:4841] TS 636960a0dafc4904a58bb65895f443b7 (127.18.129.2:40623): tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a (table test-table [id=72ecddbf97e34af6bf8738aa4b8e716f]) successfully deleted
W20260807 15:06:49.945570 21415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:35385 (error 108)
I20260807 15:06:49.945856 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 21765
I20260807 15:06:49.949764 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:49.945955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35385 (local address 127.0.0.1:58440)
0807 15:06:49.949656 (+ 3701us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:49.949718 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:35385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3673,"thread_start_us":65,"threads_started":1}
W20260807 15:06:49.949922 21415 catalog_manager.cc:4584] TS fa18c579e45c4a9588bf4427b78f1156 (127.18.129.1:35385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 754f8a8dea5b447fb9c3e6ca63ba0c2a: Network error: Client connection negotiation failed: client connection to 127.18.129.1:35385: connect: Connection refused (error 111)
I20260807 15:06:49.955497 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 21974
I20260807 15:06:49.969895 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 21390
2026-08-07T15:06:49Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5940 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:50 dist-test-slave-5bn2 krb5kdc[22432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:50 dist-test-slave-5bn2 krb5kdc[22432](info): set up 2 sockets
Aug 07 15:06:50 dist-test-slave-5bn2 krb5kdc[22432](info): commencing operation
krb5kdc: starting...
W20260807 15:06:51.101166 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:51.217317 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39839
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:39839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:51.298655 22453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:51.298946 22453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:51.298992 22453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:51.300227 22453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:51.300275 22453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:51.300299 22453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:51.300315 22453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:51.300330 22453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:51.301858 22453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:39839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:51.302132 22453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:51.302397 22453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:06:51.308118 22453 server_base.cc:1034] running on GCE node
W20260807 15:06:51.308058 22459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:51.308058 22460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:51.308077 22466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:51.308555 22453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:51.308841 22453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:51.309985 22453 hybrid_clock.cc:648] HybridClock initialized: now 1786115211309972 us; error 26 us; skew 500 ppm
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:51.312382 22453 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:51.312827 22453 webserver.cc:460] Webserver started at http://127.18.129.62:34671/ using document root <none> and password file <none>
I20260807 15:06:51.313052 22453 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:51.313103 22453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:51.313228 22453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:51.314254 22453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "fde645e622b64cf897df11b81e254ae6"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "b80d11fe74d59d4f11a1d628571c5f1a"
server_key_iv: "06bfc8e45059ae5425cf46bd87e24c9b"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.314612 22453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "fde645e622b64cf897df11b81e254ae6"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "b80d11fe74d59d4f11a1d628571c5f1a"
server_key_iv: "06bfc8e45059ae5425cf46bd87e24c9b"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.315881 22453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:51.316535 22475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.316697 22453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.316783 22453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "fde645e622b64cf897df11b81e254ae6"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "b80d11fe74d59d4f11a1d628571c5f1a"
server_key_iv: "06bfc8e45059ae5425cf46bd87e24c9b"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.316850 22453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:51.338742 22453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:51.340497 22453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:51.340658 22453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:51.344591 22453 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:39839
I20260807 15:06:51.344630 22558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:39839 every 8 connection(s)
I20260807 15:06:51.344956 22453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:51.345799 22560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:51.348124 22560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6: Bootstrap starting.
I20260807 15:06:51.348774 22560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:51.349051 22560 log.cc:824] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:51.349725 22560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6: No bootstrap required, opened a new log
I20260807 15:06:51.351394 22560 raft_consensus.cc:348] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde645e622b64cf897df11b81e254ae6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39839 } }
I20260807 15:06:51.351533 22560 raft_consensus.cc:374] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:51.351568 22560 raft_consensus.cc:729] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fde645e622b64cf897df11b81e254ae6, State: Initialized, Role: FOLLOWER
I20260807 15:06:51.351708 22560 consensus_queue.cc:260] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde645e622b64cf897df11b81e254ae6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39839 } }
I20260807 15:06:51.351788 22560 raft_consensus.cc:388] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:51.351818 22560 raft_consensus.cc:482] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:51.351867 22560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:51.351881 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 22453
I20260807 15:06:51.351969 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:51.352128 18948 external_mini_cluster.cc:1428] Setting key 92273bd45effb7653b8bfc027d367530
I20260807 15:06:51.352499 22560 raft_consensus.cc:504] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde645e622b64cf897df11b81e254ae6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39839 } }
I20260807 15:06:51.352614 22560 leader_election.cc:302] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fde645e622b64cf897df11b81e254ae6; no voters:
I20260807 15:06:51.352777 22560 leader_election.cc:288] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:51.352818 22567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:51.352952 22567 raft_consensus.cc:686] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [term 1 LEADER]: Becoming Leader. State: Replica: fde645e622b64cf897df11b81e254ae6, State: Running, Role: LEADER
I20260807 15:06:51.353000 22560 sys_catalog.cc:564] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:51.353032 22567 consensus_queue.cc:237] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde645e622b64cf897df11b81e254ae6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39839 } }
I20260807 15:06:51.353338 22567 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fde645e622b64cf897df11b81e254ae6. Latest consensus state: current_term: 1 leader_uuid: "fde645e622b64cf897df11b81e254ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde645e622b64cf897df11b81e254ae6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39839 } } }
I20260807 15:06:51.353399 22567 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:51.353546 22569 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fde645e622b64cf897df11b81e254ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde645e622b64cf897df11b81e254ae6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39839 } } }
I20260807 15:06:51.353617 22569 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:51.353904 22574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:51.354426 22574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:51.356163 22574 catalog_manager.cc:1347] Generated new cluster ID: 70c378e80ac64000a0cc901e34ede300
I20260807 15:06:51.356220 22574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:51.362579 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:51.352560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51556 (local address 127.18.129.62:39839)
0807 15:06:51.352738 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:51.352748 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:51.353717 (+ 969us) server_negotiation.cc:407] Connection header received
0807 15:06:51.353848 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:51.353856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:51.353919 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:51.354074 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:51.354983 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.356059 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.356833 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.357293 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.357388 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:51.359591 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:51.359595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:51.359596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:51.359608 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:51.361154 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:51.361468 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:51.361471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:51.361471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:51.361579 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:51.361756 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:51.361757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:51.361757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:51.361989 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:51.362132 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:51.362251 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 15:06:51.362385 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260807 15:06:51.379890 22574 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:51.380626 22574 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:51.385456 22574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fde645e622b64cf897df11b81e254ae6: Generated new TSK 0
I20260807 15:06:51.385643 22574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:06:51.416038 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39839
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:51.497040 22598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:51.497258 22598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:51.497280 22598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:51.498431 22598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:51.498472 22598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:51.498498 22598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:51.499938 22598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:51.500172 22598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:51.500383 22598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:51.505378 22609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:51.505358 22607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:51.505355 22612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:51.505604 22598 server_base.cc:1034] running on GCE node
I20260807 15:06:51.505780 22598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:51.505986 22598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:51.507109 22598 hybrid_clock.cc:648] HybridClock initialized: now 1786115211507096 us; error 29 us; skew 500 ppm
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:51.509444 22598 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:06:51.509876 22598 webserver.cc:460] Webserver started at http://127.18.129.1:40771/ using document root <none> and password file <none>
I20260807 15:06:51.510078 22598 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:51.510139 22598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:51.510267 22598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:51.511350 22598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "bf1a77112fdf410492e3a73a4d8d435a"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "c5ca6bd25bc6accc54932e9822796c43"
server_key_iv: "12f0ec772577c59ddcb0a40ec8eda314"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.511683 22598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "bf1a77112fdf410492e3a73a4d8d435a"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "c5ca6bd25bc6accc54932e9822796c43"
server_key_iv: "12f0ec772577c59ddcb0a40ec8eda314"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.512917 22598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:06:51.513618 22620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.513764 22598 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:51.513852 22598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "bf1a77112fdf410492e3a73a4d8d435a"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "c5ca6bd25bc6accc54932e9822796c43"
server_key_iv: "12f0ec772577c59ddcb0a40ec8eda314"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.513913 22598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:51.525714 22598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:51.527436 22598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:51.527568 22598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:51.527776 22598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:51.528079 22598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:51.528113 22598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.528144 22598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:51.528160 22598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.533725 22598 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:42415
I20260807 15:06:51.533771 22793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:42415 every 8 connection(s)
I20260807 15:06:51.534090 22598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:51.541078 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 22598
I20260807 15:06:51.541180 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:51.541348 18948 external_mini_cluster.cc:1428] Setting key efe041f871ec86e67eb904b208534669
I20260807 15:06:51.543069 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:51.534788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:41273 (local address 127.18.129.62:39839)
0807 15:06:51.534894 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:51.534896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:51.535066 (+ 170us) server_negotiation.cc:407] Connection header received
0807 15:06:51.535524 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:51.535525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:51.535542 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:51.535588 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:51.536475 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.536987 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.537942 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.538141 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.538190 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:51.540266 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:51.540269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:51.540269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:51.540276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:51.541743 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:51.542151 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:51.542152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:51.542153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:51.542214 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:51.542435 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:51.542435 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:51.542436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:51.542559 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:51.542616 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:51.542888 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 15:06:51.542978 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:06:51.543227 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:51.534760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39839 (local address 127.18.129.1:41273)
0807 15:06:51.534947 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:51.534985 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:51.535449 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:51.535655 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:51.535658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:51.536218 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:51.536408 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:51.536412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.537068 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.537069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:51.537878 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:51.537881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.538203 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.538204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:51.538302 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:51.538761 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:51.538768 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:51.540171 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:51.541814 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:51.541822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:51.541824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:51.542078 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:51.542271 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:51.542271 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:51.542271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:51.542376 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:51.542615 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:51.542623 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:51.542749 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 15:06:51.542836 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:06:51.543011 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
I20260807 15:06:51.543672 22794 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39839
I20260807 15:06:51.543769 22794 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:51.543944 22794 heartbeater.cc:511] Master 127.18.129.62:39839 requested a full tablet report, sending...
I20260807 15:06:51.544445 22505 ts_manager.cc:194] Registered new tserver with Master: bf1a77112fdf410492e3a73a4d8d435a (127.18.129.1:42415)
I20260807 15:06:51.545075 22505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:41273
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:06:51.567739 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39839
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:51.650046 22799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:51.650285 22799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:51.650307 22799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:51.651469 22799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:51.651521 22799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:51.651551 22799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:51.653079 22799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:51.653332 22799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:51.653564 22799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:51.658680 22810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:51.658694 22809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:51.658687 22815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:51.658720 22799 server_base.cc:1034] running on GCE node
I20260807 15:06:51.659165 22799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:51.659382 22799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:51.660516 22799 hybrid_clock.cc:648] HybridClock initialized: now 1786115211660503 us; error 29 us; skew 500 ppm
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:51.662914 22799 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:06:51.663321 22799 webserver.cc:460] Webserver started at http://127.18.129.2:38551/ using document root <none> and password file <none>
I20260807 15:06:51.663532 22799 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:51.663590 22799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:51.663722 22799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:51.664763 22799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "de46e77801fd460ebda3a06813583c30"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "05c682692da61217a0b2987246e6e75e"
server_key_iv: "203f547e0fc7b292a2dc18388d935ce6"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.665109 22799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "de46e77801fd460ebda3a06813583c30"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "05c682692da61217a0b2987246e6e75e"
server_key_iv: "203f547e0fc7b292a2dc18388d935ce6"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.666368 22799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:06:51.667064 22829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.667210 22799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:06:51.667295 22799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "de46e77801fd460ebda3a06813583c30"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "05c682692da61217a0b2987246e6e75e"
server_key_iv: "203f547e0fc7b292a2dc18388d935ce6"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.667357 22799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:51.697423 22799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:51.699234 22799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:51.699375 22799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:51.699609 22799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:51.699920 22799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:51.699955 22799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.699986 22799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:51.700007 22799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.705734 22799 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:33205
I20260807 15:06:51.705798 22998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:33205 every 8 connection(s)
I20260807 15:06:51.706089 22799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:51.713022 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 22799
I20260807 15:06:51.713124 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:51.713292 18948 external_mini_cluster.cc:1428] Setting key 2feca843078c383d8a98b2586ccccd74
I20260807 15:06:51.714816 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:51.706741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:60833 (local address 127.18.129.62:39839)
0807 15:06:51.706822 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:51.706824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:51.707016 (+ 192us) server_negotiation.cc:407] Connection header received
0807 15:06:51.707487 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:51.707488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:51.707498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:51.707549 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:51.708440 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.708937 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.709838 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.710015 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.710062 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:51.712150 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:51.712152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:51.712152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:51.712160 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:51.713480 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:51.713908 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:51.713910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:51.713910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:51.713973 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:51.714197 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:51.714198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:51.714198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:51.714315 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:51.714372 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:51.714636 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 15:06:51.714728 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:06:51.714998 23003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:51.706714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39839 (local address 127.18.129.2:60833)
0807 15:06:51.706890 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:51.706938 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:51.707414 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:51.707606 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:51.707614 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:51.708173 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:51.708371 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:51.708377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.708996 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.708997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:51.709778 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:51.709781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.710081 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.710082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:51.710224 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:51.710653 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:51.710662 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:51.712049 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:51.713553 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:51.713561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:51.713563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:51.713849 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:51.714026 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:51.714026 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:51.714027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:51.714138 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:51.714368 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:51.714375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:51.714512 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 15:06:51.714605 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:06:51.714761 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260807 15:06:51.715411 22999 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39839
I20260807 15:06:51.715509 22999 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:51.715706 22999 heartbeater.cc:511] Master 127.18.129.62:39839 requested a full tablet report, sending...
I20260807 15:06:51.716090 22505 ts_manager.cc:194] Registered new tserver with Master: de46e77801fd460ebda3a06813583c30 (127.18.129.2:33205)
I20260807 15:06:51.716569 22505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:60833
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:06:51.737484 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39839
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:51.818667 23007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:51.818915 23007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:51.818949 23007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:51.820150 23007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:51.820202 23007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:51.820236 23007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:51.821679 23007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:51.821928 23007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:51.822137 23007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:51.827194 23015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:51.827239 23016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:51.827190 23019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:51.827436 23007 server_base.cc:1034] running on GCE node
I20260807 15:06:51.827667 23007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:51.827894 23007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:51.829035 23007 hybrid_clock.cc:648] HybridClock initialized: now 1786115211829018 us; error 30 us; skew 500 ppm
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:51.831529 23007 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:06:51.831991 23007 webserver.cc:460] Webserver started at http://127.18.129.3:40925/ using document root <none> and password file <none>
I20260807 15:06:51.832197 23007 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:51.832247 23007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:51.832376 23007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:51.833417 23007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "509a6db41f6647a39b37c7812c9a78d7"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "a63411d9b4717d82ea8d2c2dd61b4a3c"
server_key_iv: "67c8b94e465388efc70e3d1633cf46c7"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.833766 23007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "509a6db41f6647a39b37c7812c9a78d7"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "a63411d9b4717d82ea8d2c2dd61b4a3c"
server_key_iv: "67c8b94e465388efc70e3d1633cf46c7"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.835103 23007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:06:51.835853 23029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.835996 23007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:51.836083 23007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "509a6db41f6647a39b37c7812c9a78d7"
format_stamp: "Formatted at 2026-08-07 15:06:51 on dist-test-slave-5bn2"
server_key: "a63411d9b4717d82ea8d2c2dd61b4a3c"
server_key_iv: "67c8b94e465388efc70e3d1633cf46c7"
server_key_version: "encryptionkey@0"
I20260807 15:06:51.836154 23007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:51.857141 23007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:51.859025 23007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:51.859163 23007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:51.859397 23007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:51.859715 23007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:51.859750 23007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.859776 23007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:51.859798 23007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:51.865586 23007 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:45265
I20260807 15:06:51.865634 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:45265 every 8 connection(s)
I20260807 15:06:51.865945 23007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:06:51 dist-test-slave-5bn2 krb5kdc[22432](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115211, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:51.872417 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 23007
I20260807 15:06:51.872505 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:51.872668 18948 external_mini_cluster.cc:1428] Setting key 8c1e3bf39e5b57a8c0a70607fc316016
I20260807 15:06:51.875168 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:51.866695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:43049 (local address 127.18.129.62:39839)
0807 15:06:51.866753 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:51.866755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:51.867009 (+ 254us) server_negotiation.cc:407] Connection header received
0807 15:06:51.867551 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:51.867552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:51.867567 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:51.867619 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:51.868611 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.869103 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.870125 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.870315 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.870380 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:51.872519 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:51.872520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:51.872521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:51.872529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:51.873985 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:51.874349 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:51.874350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:51.874350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:51.874415 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:51.874619 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:51.874619 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:51.874619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:51.874720 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:51.874766 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:51.875035 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 15:06:51.875102 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:06:51.875361 23213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:51.866655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39839 (local address 127.18.129.3:43049)
0807 15:06:51.866842 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:51.866928 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:51.867449 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:51.867693 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:51.867697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:51.868311 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:51.868516 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:51.868521 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.869186 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.869187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:51.870034 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:51.870037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:51.870388 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:51.870389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:51.870523 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:51.871076 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:51.871084 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:51.872426 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:51.874056 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:51.874062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:51.874064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:51.874290 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:51.874466 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:51.874466 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:51.874467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:51.874571 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:51.874764 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:51.874767 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:51.874920 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 15:06:51.875008 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:06:51.875148 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260807 15:06:51.875759 23211 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39839
I20260807 15:06:51.875844 23211 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:51.876016 23211 heartbeater.cc:511] Master 127.18.129.62:39839 requested a full tablet report, sending...
I20260807 15:06:51.876377 22505 ts_manager.cc:194] Registered new tserver with Master: 509a6db41f6647a39b37c7812c9a78d7 (127.18.129.3:45265)
I20260807 15:06:51.876717 22505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:43049
I20260807 15:06:51.888855 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:06:51.902272 22565 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:06:51.901376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51558 (local address 127.18.129.62:39839)
0807 15:06:51.901449 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:51.901451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:51.901532 (+ 81us) server_negotiation.cc:407] Connection header received
0807 15:06:51.901601 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:51.901602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:51.901612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:51.901659 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:51.902213 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51558: BlockingRecv error: recv got EOF from 127.0.0.1:51558 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:06:51.903275 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 22598
I20260807 15:06:51.909787 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 22799
I20260807 15:06:51.940740 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 23007
I20260807 15:06:51.946053 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 22453
2026-08-07T15:06:51Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1975 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:52 dist-test-slave-5bn2 krb5kdc[23227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:52 dist-test-slave-5bn2 krb5kdc[23227](info): set up 2 sockets
Aug 07 15:06:52 dist-test-slave-5bn2 krb5kdc[23227](info): commencing operation
krb5kdc: starting...
W20260807 15:06:53.083195 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:53 dist-test-slave-5bn2 krb5kdc[23227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:53.201282 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:32977
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:32977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:53.281977 23241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:53.282218 23241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:53.282248 23241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:53.283804 23241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:53.283859 23241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:53.283877 23241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:53.283895 23241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:53.283917 23241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:53.285497 23241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:32977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:32977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:53.285760 23241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:53.285987 23241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:53.291041 23260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:53.291077 23241 server_base.cc:1034] running on GCE node
W20260807 15:06:53.291042 23255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:53.291072 23249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:53.291488 23241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:53.291795 23241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:53.292940 23241 hybrid_clock.cc:648] HybridClock initialized: now 1786115213292923 us; error 35 us; skew 500 ppm
Aug 07 15:06:53 dist-test-slave-5bn2 krb5kdc[23227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115213, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:53.299628 23241 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:53.300138 23241 webserver.cc:460] Webserver started at http://127.18.129.62:37199/ using document root <none> and password file <none>
I20260807 15:06:53.300362 23241 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:53.300406 23241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:53.300511 23241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:53.301577 23241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "f4ed9ff787f543b384ffd1771848a561"
format_stamp: "Formatted at 2026-08-07 15:06:53 on dist-test-slave-5bn2"
server_key: "dba67403b6950a99a7717e6633534934"
server_key_iv: "517265a21dc44768637427b416e5e56b"
server_key_version: "encryptionkey@0"
I20260807 15:06:53.301878 23241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "f4ed9ff787f543b384ffd1771848a561"
format_stamp: "Formatted at 2026-08-07 15:06:53 on dist-test-slave-5bn2"
server_key: "dba67403b6950a99a7717e6633534934"
server_key_iv: "517265a21dc44768637427b416e5e56b"
server_key_version: "encryptionkey@0"
I20260807 15:06:53.305069 23241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 15:06:53.305850 23271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:53.306021 23241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:06:53.306130 23241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "f4ed9ff787f543b384ffd1771848a561"
format_stamp: "Formatted at 2026-08-07 15:06:53 on dist-test-slave-5bn2"
server_key: "dba67403b6950a99a7717e6633534934"
server_key_iv: "517265a21dc44768637427b416e5e56b"
server_key_version: "encryptionkey@0"
I20260807 15:06:53.306192 23241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:53.330443 23241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:53.332564 23241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:53.332691 23241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:53.336817 23241 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:32977
I20260807 15:06:53.336859 23350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:32977 every 8 connection(s)
I20260807 15:06:53.337406 23241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:53.338148 23351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:53.340637 23351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561: Bootstrap starting.
I20260807 15:06:53.341234 23351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:53.341512 23351 log.cc:824] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:53.342191 23351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561: No bootstrap required, opened a new log
I20260807 15:06:53.343837 23351 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed9ff787f543b384ffd1771848a561" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 32977 } }
I20260807 15:06:53.343983 23351 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:53.344017 23351 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4ed9ff787f543b384ffd1771848a561, State: Initialized, Role: FOLLOWER
I20260807 15:06:53.344158 23351 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed9ff787f543b384ffd1771848a561" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 32977 } }
I20260807 15:06:53.344241 23351 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:53.344273 23351 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:53.344323 23351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:53.345216 23351 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed9ff787f543b384ffd1771848a561" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 32977 } }
I20260807 15:06:53.345315 23351 leader_election.cc:302] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4ed9ff787f543b384ffd1771848a561; no voters:
I20260807 15:06:53.345484 23351 leader_election.cc:288] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:53.345538 23355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:53.345690 23355 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [term 1 LEADER]: Becoming Leader. State: Replica: f4ed9ff787f543b384ffd1771848a561, State: Running, Role: LEADER
I20260807 15:06:53.345738 23351 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:53.345782 23355 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed9ff787f543b384ffd1771848a561" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 32977 } }
I20260807 15:06:53.346139 23357 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4ed9ff787f543b384ffd1771848a561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed9ff787f543b384ffd1771848a561" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 32977 } } }
I20260807 15:06:53.346230 23357 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:53.346342 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 23241
I20260807 15:06:53.346436 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:53.346414 23358 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4ed9ff787f543b384ffd1771848a561. Latest consensus state: current_term: 1 leader_uuid: "f4ed9ff787f543b384ffd1771848a561" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed9ff787f543b384ffd1771848a561" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 32977 } } }
I20260807 15:06:53.346464 23358 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:53.346590 23373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:53.346649 18948 external_mini_cluster.cc:1428] Setting key f18c5e299cbf20b38d5b544c1979631e
I20260807 15:06:53.347167 23373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:53.348922 23373 catalog_manager.cc:1347] Generated new cluster ID: 14dd9a28e7b242159d2de76cc49a0803
I20260807 15:06:53.348965 23373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:06:53 dist-test-slave-5bn2 krb5kdc[23227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:53.356554 23373 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:53.357182 23373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:53.358343 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:53.347299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.18.129.62:32977)
0807 15:06:53.347526 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:53.347537 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:53.347581 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:06:53.347897 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:53.347904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:53.347968 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:53.348111 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:53.348894 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:53.350310 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:53.351353 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:53.352011 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:53.352137 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:53.354745 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:53.354749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:53.354750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:53.354763 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:53.356720 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:53.357104 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:53.357107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:53.357108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:53.357238 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:53.357460 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:53.357461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:53.357462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:53.357732 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:53.357881 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:53.357975 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 15:06:53.358122 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260807 15:06:53.359495 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:06:53.363915 23373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4ed9ff787f543b384ffd1771848a561: Generated new TSK 0
I20260807 15:06:53.364104 23373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:53.378300 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:53.374612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60098 (local address 127.18.129.62:32977)
0807 15:06:53.374669 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:53.374672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:53.374777 (+ 105us) server_negotiation.cc:407] Connection header received
0807 15:06:53.374829 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:53.374830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:53.374842 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:53.374931 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:53.375271 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:53.376122 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:53.376787 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:53.377125 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:53.377188 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:53.377782 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:53.377783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:53.377784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:53.378030 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:53.378060 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:53.378067 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:53.378128 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:53.378184 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:06:53.378213 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:06:53.380730 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 23241
2026-08-07T15:06:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1437 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:53 dist-test-slave-5bn2 krb5kdc[23418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:53 dist-test-slave-5bn2 krb5kdc[23418](info): set up 2 sockets
Aug 07 15:06:53 dist-test-slave-5bn2 krb5kdc[23418](info): commencing operation
krb5kdc: starting...
W20260807 15:06:54.535933 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:54 dist-test-slave-5bn2 krb5kdc[23418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:54.655539 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39869
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:39869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:54.736716 23440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:54.736967 23440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:54.737000 23440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:54.738169 23440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:54.738215 23440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:54.738234 23440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:54.738250 23440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:54.738266 23440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:54.739812 23440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:39869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:54.740075 23440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:54.740284 23440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:54.745126 23448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:54.745129 23450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:54.745129 23453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:54.745298 23440 server_base.cc:1034] running on GCE node
I20260807 15:06:54.745595 23440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:54.745849 23440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:54.746977 23440 hybrid_clock.cc:648] HybridClock initialized: now 1786115214746952 us; error 36 us; skew 500 ppm
Aug 07 15:06:54 dist-test-slave-5bn2 krb5kdc[23418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115214, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:54.749408 23440 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:54.749799 23440 webserver.cc:460] Webserver started at http://127.18.129.62:41275/ using document root <none> and password file <none>
I20260807 15:06:54.750015 23440 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:54.750067 23440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:54.750173 23440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:54.751243 23440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "055f4fa2d0b845fb8647b6960e41f726"
format_stamp: "Formatted at 2026-08-07 15:06:54 on dist-test-slave-5bn2"
server_key: "d802b0fea3a4c0bb38e3fd6fc002f762"
server_key_iv: "e4bb4cc147ffb47037f1b70f0b5c5e5a"
server_key_version: "encryptionkey@0"
I20260807 15:06:54.751577 23440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "055f4fa2d0b845fb8647b6960e41f726"
format_stamp: "Formatted at 2026-08-07 15:06:54 on dist-test-slave-5bn2"
server_key: "d802b0fea3a4c0bb38e3fd6fc002f762"
server_key_iv: "e4bb4cc147ffb47037f1b70f0b5c5e5a"
server_key_version: "encryptionkey@0"
I20260807 15:06:54.752867 23440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:06:54.753504 23465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:54.753649 23440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:54.753732 23440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "055f4fa2d0b845fb8647b6960e41f726"
format_stamp: "Formatted at 2026-08-07 15:06:54 on dist-test-slave-5bn2"
server_key: "d802b0fea3a4c0bb38e3fd6fc002f762"
server_key_iv: "e4bb4cc147ffb47037f1b70f0b5c5e5a"
server_key_version: "encryptionkey@0"
I20260807 15:06:54.753788 23440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:54.772002 23440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:54.774161 23440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:54.774276 23440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:54.778141 23440 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:39869
I20260807 15:06:54.778191 23540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:39869 every 8 connection(s)
I20260807 15:06:54.778507 23440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:54.779646 23542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:54.780376 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 23440
I20260807 15:06:54.780465 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:54.780628 18948 external_mini_cluster.cc:1428] Setting key f2289ad4898eea9112c9d745ea28dd48
I20260807 15:06:54.782652 23542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726: Bootstrap starting.
I20260807 15:06:54.783402 23542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:54.783725 23542 log.cc:824] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:54.784415 23542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726: No bootstrap required, opened a new log
I20260807 15:06:54.786590 23542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055f4fa2d0b845fb8647b6960e41f726" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39869 } }
I20260807 15:06:54.786727 23542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:54.786753 23542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055f4fa2d0b845fb8647b6960e41f726, State: Initialized, Role: FOLLOWER
I20260807 15:06:54.786906 23542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055f4fa2d0b845fb8647b6960e41f726" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39869 } }
I20260807 15:06:54.787009 23542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:54.787048 23542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:54.787096 23542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 15:06:54 dist-test-slave-5bn2 krb5kdc[23418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:54.787732 23542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055f4fa2d0b845fb8647b6960e41f726" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39869 } }
I20260807 15:06:54.787827 23542 leader_election.cc:302] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 055f4fa2d0b845fb8647b6960e41f726; no voters:
I20260807 15:06:54.788013 23542 leader_election.cc:288] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:54.788057 23551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:54.788228 23551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [term 1 LEADER]: Becoming Leader. State: Replica: 055f4fa2d0b845fb8647b6960e41f726, State: Running, Role: LEADER
I20260807 15:06:54.788277 23542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:54.788627 23551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055f4fa2d0b845fb8647b6960e41f726" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39869 } }
I20260807 15:06:54.789171 23552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 055f4fa2d0b845fb8647b6960e41f726. Latest consensus state: current_term: 1 leader_uuid: "055f4fa2d0b845fb8647b6960e41f726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055f4fa2d0b845fb8647b6960e41f726" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39869 } } }
I20260807 15:06:54.789167 23553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "055f4fa2d0b845fb8647b6960e41f726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055f4fa2d0b845fb8647b6960e41f726" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39869 } } }
I20260807 15:06:54.789279 23553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:54.789484 23563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:54.790009 23563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:54.789409 23552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:54.791815 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:54.781131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.18.129.62:39869)
0807 15:06:54.781318 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:54.781328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:54.781370 (+ 42us) server_negotiation.cc:407] Connection header received
0807 15:06:54.781888 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:54.781897 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:54.781964 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:54.782127 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:54.782686 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:54.784041 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:54.784842 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:54.785463 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:54.785588 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:54.787932 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:54.787938 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:54.787939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:54.787951 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:54.790088 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:54.790513 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:54.790516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:54.790516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:54.790640 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:54.790918 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:54.790919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:54.790920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:54.791164 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:54.791322 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:54.791469 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 15:06:54.791599 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260807 15:06:54.795280 23563 catalog_manager.cc:1347] Generated new cluster ID: 35e10d0ccf6d401f89e82c223d00bf7e
I20260807 15:06:54.795346 23563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:54.803213 23563 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:54.803931 23563 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:54.813980 23563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 055f4fa2d0b845fb8647b6960e41f726: Generated new TSK 0
I20260807 15:06:54.814177 23563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:06:54.839968 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39869
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:54.921871 23586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:54.922104 23586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:54.922139 23586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:54.923329 23586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:54.923374 23586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:54.923409 23586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:54.925123 23586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:54.925388 23586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:54.925649 23586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:54.930629 23594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:54.930637 23598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:54.930662 23592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:54.930884 23586 server_base.cc:1034] running on GCE node
I20260807 15:06:54.931092 23586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:54.931313 23586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:54.932446 23586 hybrid_clock.cc:648] HybridClock initialized: now 1786115214932433 us; error 29 us; skew 500 ppm
Aug 07 15:06:54 dist-test-slave-5bn2 krb5kdc[23418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115214, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:54.935000 23586 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:06:54.935451 23586 webserver.cc:460] Webserver started at http://127.18.129.1:37359/ using document root <none> and password file <none>
I20260807 15:06:54.935642 23586 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:54.935683 23586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:54.935774 23586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:54.936815 23586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "1083bf168b42408fb6e3e54d641820c4"
format_stamp: "Formatted at 2026-08-07 15:06:54 on dist-test-slave-5bn2"
server_key: "d95093466f2b87e8bc083f329e49a5cc"
server_key_iv: "79d504bcc3892664632a7f570bffac55"
server_key_version: "encryptionkey@0"
I20260807 15:06:54.937155 23586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "1083bf168b42408fb6e3e54d641820c4"
format_stamp: "Formatted at 2026-08-07 15:06:54 on dist-test-slave-5bn2"
server_key: "d95093466f2b87e8bc083f329e49a5cc"
server_key_iv: "79d504bcc3892664632a7f570bffac55"
server_key_version: "encryptionkey@0"
I20260807 15:06:54.938367 23586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:54.939034 23610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:54.939182 23586 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:54.939265 23586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "1083bf168b42408fb6e3e54d641820c4"
format_stamp: "Formatted at 2026-08-07 15:06:54 on dist-test-slave-5bn2"
server_key: "d95093466f2b87e8bc083f329e49a5cc"
server_key_iv: "79d504bcc3892664632a7f570bffac55"
server_key_version: "encryptionkey@0"
I20260807 15:06:54.939322 23586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:54.954640 23586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:54.956769 23586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:54.956902 23586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:54.957118 23586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:54.957459 23586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:54.957499 23586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:54.957530 23586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:54.957545 23586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:54.962997 23586 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:39855
I20260807 15:06:54.963035 23791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:39855 every 8 connection(s)
I20260807 15:06:54.963348 23586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:06:54.965682 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 23586
I20260807 15:06:54.965790 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:54.965986 18948 external_mini_cluster.cc:1428] Setting key f37ab96c4501adc296221518b4638fe6
Aug 07 15:06:54 dist-test-slave-5bn2 krb5kdc[23418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115214, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:54.972847 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:54.964030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:43283 (local address 127.18.129.62:39869)
0807 15:06:54.964107 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:54.964109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:54.964272 (+ 163us) server_negotiation.cc:407] Connection header received
0807 15:06:54.964761 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:54.964762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:54.964771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:54.964821 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:54.965736 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:54.966379 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:54.967458 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:54.967669 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:54.967724 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:54.970121 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:54.970122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:54.970122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:54.970129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:54.971455 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:54.971923 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:54.971924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:54.971924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:54.971976 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:54.972244 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:54.972244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:54.972245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:54.972343 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:54.972389 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:54.972708 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 15:06:54.972779 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:06:54.973119 23794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:54.964000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39869 (local address 127.18.129.1:43283)
0807 15:06:54.964160 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:54.964197 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:54.964685 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:54.964874 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:54.964877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:54.965450 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:54.965650 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:54.965656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:54.966473 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:54.966474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:54.967368 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:54.967371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:54.967741 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:54.967742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:54.967857 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:54.968382 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:54.968392 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:54.969998 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:54.971541 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:54.971551 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:54.971553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:54.971851 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:54.972040 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:54.972041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:54.972041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:54.972185 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:54.972398 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:54.972404 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:54.972574 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 15:06:54.972696 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 15:06:54.972867 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260807 15:06:54.973673 23792 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39869
I20260807 15:06:54.973760 23792 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:54.973927 23792 heartbeater.cc:511] Master 127.18.129.62:39869 requested a full tablet report, sending...
I20260807 15:06:54.974457 23488 ts_manager.cc:194] Registered new tserver with Master: 1083bf168b42408fb6e3e54d641820c4 (127.18.129.1:39855)
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
I20260807 15:06:54.975126 23488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:43283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:06:54.991612 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39869
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:55.073746 23797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:55.073962 23797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:55.073992 23797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:55.075173 23797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:55.075214 23797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:55.075248 23797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:55.076697 23797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:55.076944 23797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:55.077150 23797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:55.081998 23807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:55.082005 23805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:55.082057 23809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:55.082252 23797 server_base.cc:1034] running on GCE node
I20260807 15:06:55.082455 23797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:55.082676 23797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:55.083812 23797 hybrid_clock.cc:648] HybridClock initialized: now 1786115215083800 us; error 29 us; skew 500 ppm
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[23418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115215, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:55.086591 23797 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:06:55.087077 23797 webserver.cc:460] Webserver started at http://127.18.129.2:46365/ using document root <none> and password file <none>
I20260807 15:06:55.087306 23797 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:55.087354 23797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:55.087483 23797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:55.088500 23797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "48f59f54a15b4e418a54500756efd166"
format_stamp: "Formatted at 2026-08-07 15:06:55 on dist-test-slave-5bn2"
server_key: "52acf62dd1deab129ea9eaa6bb358696"
server_key_iv: "4fd5ef3cb44d3553342f4389e48f7e49"
server_key_version: "encryptionkey@0"
I20260807 15:06:55.088929 23797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "48f59f54a15b4e418a54500756efd166"
format_stamp: "Formatted at 2026-08-07 15:06:55 on dist-test-slave-5bn2"
server_key: "52acf62dd1deab129ea9eaa6bb358696"
server_key_iv: "4fd5ef3cb44d3553342f4389e48f7e49"
server_key_version: "encryptionkey@0"
I20260807 15:06:55.090234 23797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:55.090935 23822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:55.091080 23797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:55.091169 23797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "48f59f54a15b4e418a54500756efd166"
format_stamp: "Formatted at 2026-08-07 15:06:55 on dist-test-slave-5bn2"
server_key: "52acf62dd1deab129ea9eaa6bb358696"
server_key_iv: "4fd5ef3cb44d3553342f4389e48f7e49"
server_key_version: "encryptionkey@0"
I20260807 15:06:55.091238 23797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:55.101167 23797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:55.103245 23797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:55.103382 23797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:55.103585 23797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:55.103873 23797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:55.103905 23797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:55.103933 23797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:55.103951 23797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:55.109604 23797 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:34249
I20260807 15:06:55.109642 23984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:34249 every 8 connection(s)
I20260807 15:06:55.109939 23797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[23418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115215, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:55.116389 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 23797
I20260807 15:06:55.116503 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:55.116677 18948 external_mini_cluster.cc:1428] Setting key 7886dc07fbf48138b483c08c911facbc
I20260807 15:06:55.118760 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:55.110595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:39831 (local address 127.18.129.62:39869)
0807 15:06:55.110643 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:55.110645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:55.110832 (+ 187us) server_negotiation.cc:407] Connection header received
0807 15:06:55.111363 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:55.111364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:55.111377 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:55.111425 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:55.112381 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.112870 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.113839 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.114031 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.114092 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:55.116050 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:55.116052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:55.116052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:55.116060 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:55.117554 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:55.117905 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:55.117907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:55.117907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:55.117975 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:55.118192 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:55.118192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:55.118193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:55.118319 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:55.118381 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:55.118589 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 15:06:55.118671 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:06:55.118875 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:55.110568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39869 (local address 127.18.129.2:39831)
0807 15:06:55.110720 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:55.110756 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:55.111266 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:55.111503 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:55.111507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:55.112072 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:55.112284 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:55.112290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.112951 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.112952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:55.113756 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:55.113759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.114110 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.114111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:55.114236 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:55.114647 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:55.114654 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:55.115957 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:55.117630 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:55.117636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:55.117638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:55.117843 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:55.118034 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:55.118034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:55.118035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:55.118142 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:55.118371 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:55.118374 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:55.118482 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 15:06:55.118557 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 15:06:55.118683 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260807 15:06:55.119354 23986 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39869
I20260807 15:06:55.119439 23986 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:55.119616 23986 heartbeater.cc:511] Master 127.18.129.62:39869 requested a full tablet report, sending...
I20260807 15:06:55.119968 23488 ts_manager.cc:194] Registered new tserver with Master: 48f59f54a15b4e418a54500756efd166 (127.18.129.2:34249)
I20260807 15:06:55.120329 23488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:39831
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:06:55.141371 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39869
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:55.222360 23991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:55.222572 23991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:55.222594 23991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:55.223695 23991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:55.223729 23991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:55.223753 23991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:55.225153 23991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:55.225342 23991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:55.225517 23991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:55.230415 24002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:55.230489 24000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:55.230501 23998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:55.230659 23991 server_base.cc:1034] running on GCE node
I20260807 15:06:55.230873 23991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:55.231122 23991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:55.232263 23991 hybrid_clock.cc:648] HybridClock initialized: now 1786115215232246 us; error 26 us; skew 500 ppm
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[23418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115215, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:55.234675 23991 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:06:55.235124 23991 webserver.cc:460] Webserver started at http://127.18.129.3:34195/ using document root <none> and password file <none>
I20260807 15:06:55.235339 23991 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:55.235387 23991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:55.235493 23991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:55.236667 23991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "89036632f8da4a1887482f17cd4272be"
format_stamp: "Formatted at 2026-08-07 15:06:55 on dist-test-slave-5bn2"
server_key: "fe4f2c26cba4d3022d415f508837a900"
server_key_iv: "36372022d891a3999d3a95dca45548f2"
server_key_version: "encryptionkey@0"
I20260807 15:06:55.237035 23991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "89036632f8da4a1887482f17cd4272be"
format_stamp: "Formatted at 2026-08-07 15:06:55 on dist-test-slave-5bn2"
server_key: "fe4f2c26cba4d3022d415f508837a900"
server_key_iv: "36372022d891a3999d3a95dca45548f2"
server_key_version: "encryptionkey@0"
I20260807 15:06:55.238425 23991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:55.239218 24013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:55.239382 23991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:55.239471 23991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "89036632f8da4a1887482f17cd4272be"
format_stamp: "Formatted at 2026-08-07 15:06:55 on dist-test-slave-5bn2"
server_key: "fe4f2c26cba4d3022d415f508837a900"
server_key_iv: "36372022d891a3999d3a95dca45548f2"
server_key_version: "encryptionkey@0"
I20260807 15:06:55.239544 23991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:55.248889 23991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:55.250957 23991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:55.251093 23991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:55.251298 23991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:55.251617 23991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:55.251658 23991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:55.251685 23991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:55.251703 23991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:55.257594 23991 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:40285
I20260807 15:06:55.257632 24178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:40285 every 8 connection(s)
I20260807 15:06:55.257928 23991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[23418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115215, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:55.266402 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 23991
I20260807 15:06:55.266506 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:55.266716 18948 external_mini_cluster.cc:1428] Setting key d465060ce18ef928076b757aa21d832a
I20260807 15:06:55.266767 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:55.258567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:52711 (local address 127.18.129.62:39869)
0807 15:06:55.258623 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:55.258625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:55.258821 (+ 196us) server_negotiation.cc:407] Connection header received
0807 15:06:55.259343 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:55.259344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:55.259355 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:55.259393 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:55.260267 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.260752 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.261821 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.261995 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.262038 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:55.264115 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:55.264116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:55.264117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:55.264123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:55.265493 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:55.265856 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:55.265856 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:55.265857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:55.265925 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:55.266114 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:55.266114 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:55.266115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:55.266221 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:55.266269 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:55.266619 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 15:06:55.266696 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:06:55.266981 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:55.258540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39869 (local address 127.18.129.3:52711)
0807 15:06:55.258696 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:55.258731 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:55.259247 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:55.259451 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:55.259455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:55.260004 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:55.260200 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:55.260205 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.260836 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.260837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:55.261754 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:55.261757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.262049 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.262049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:55.262145 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:55.262613 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:55.262621 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:55.264025 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:55.265573 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:55.265580 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:55.265582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:55.265791 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:55.265977 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:55.265978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:55.265978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:55.266069 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:55.266274 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:55.266280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:55.266446 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 15:06:55.266578 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 15:06:55.266749 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260807 15:06:55.267429 24179 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39869
I20260807 15:06:55.267534 24179 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:55.267727 24179 heartbeater.cc:511] Master 127.18.129.62:39869 requested a full tablet report, sending...
I20260807 15:06:55.268095 23488 ts_manager.cc:194] Registered new tserver with Master: 89036632f8da4a1887482f17cd4272be (127.18.129.3:40285)
I20260807 15:06:55.268483 23488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:52711
I20260807 15:06:55.278581 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[23418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115215, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[23418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115215, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:55.301770 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:55.295063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.18.129.62:39869)
0807 15:06:55.295120 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:55.295122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:55.295216 (+ 94us) server_negotiation.cc:407] Connection header received
0807 15:06:55.295291 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:55.295292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:55.295303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:55.295348 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:55.295883 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.296370 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.297075 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:55.297275 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:55.297329 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:55.299459 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:55.299461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:55.299461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:55.299470 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:55.300720 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:55.301026 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:55.301027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:55.301028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:55.301082 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:55.301248 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:55.301249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:55.301249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:55.301395 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:55.301487 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:55.301598 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 15:06:55.301674 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260807 15:06:55.301986 23488 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48388
I20260807 15:06:55.302347 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 23586
I20260807 15:06:55.323060 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 23797
I20260807 15:06:55.336696 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 23991
I20260807 15:06:55.353087 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 23440
2026-08-07T15:06:55Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1967 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[24204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[24204](info): set up 2 sockets
Aug 07 15:06:55 dist-test-slave-5bn2 krb5kdc[24204](info): commencing operation
krb5kdc: starting...
W20260807 15:06:56.476575 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:56 dist-test-slave-5bn2 krb5kdc[24204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:56.598253 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45853
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:42769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:45853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:56.680178 24222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:56.680382 24222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:56.680411 24222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:56.681541 24222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:56.681582 24222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:56.681600 24222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:56.681617 24222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:56.681631 24222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:56.683171 24222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:42769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:45853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:56.683413 24222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:56.683621 24222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:56.688442 24233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:56.688438 24232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:56.688674 24222 server_base.cc:1034] running on GCE node
W20260807 15:06:56.688433 24237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:56.689000 24222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:56.689270 24222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:56.690410 24222 hybrid_clock.cc:648] HybridClock initialized: now 1786115216690391 us; error 41 us; skew 500 ppm
Aug 07 15:06:56 dist-test-slave-5bn2 krb5kdc[24204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115216, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:56.692960 24222 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:56.693352 24222 webserver.cc:460] Webserver started at http://127.18.129.62:37355/ using document root <none> and password file <none>
I20260807 15:06:56.693580 24222 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:56.693630 24222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:56.693763 24222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:56.694778 24222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "b673735580ce466ca0e443eb4a69545a"
format_stamp: "Formatted at 2026-08-07 15:06:56 on dist-test-slave-5bn2"
server_key: "9a3674e0abe0eb962eb882615d11d4ed"
server_key_iv: "3b31f6603c284c3b3df747e1748147b0"
server_key_version: "encryptionkey@0"
I20260807 15:06:56.695150 24222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "b673735580ce466ca0e443eb4a69545a"
format_stamp: "Formatted at 2026-08-07 15:06:56 on dist-test-slave-5bn2"
server_key: "9a3674e0abe0eb962eb882615d11d4ed"
server_key_iv: "3b31f6603c284c3b3df747e1748147b0"
server_key_version: "encryptionkey@0"
I20260807 15:06:56.696362 24222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:56.697005 24249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:56.697165 24222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:56.697249 24222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "b673735580ce466ca0e443eb4a69545a"
format_stamp: "Formatted at 2026-08-07 15:06:56 on dist-test-slave-5bn2"
server_key: "9a3674e0abe0eb962eb882615d11d4ed"
server_key_iv: "3b31f6603c284c3b3df747e1748147b0"
server_key_version: "encryptionkey@0"
I20260807 15:06:56.697315 24222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:56.751741 24222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:56.753963 24222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:56.754113 24222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:56.758109 24222 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:45853
I20260807 15:06:56.758152 24328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:45853 every 8 connection(s)
I20260807 15:06:56.758646 24222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:56.759316 24329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:56.761549 24329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a: Bootstrap starting.
I20260807 15:06:56.762087 24329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:56.762336 24329 log.cc:824] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:56.762977 24329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a: No bootstrap required, opened a new log
I20260807 15:06:56.763671 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 24222
I20260807 15:06:56.763761 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:56.763923 18948 external_mini_cluster.cc:1428] Setting key b01c5eca81cac1bc0492a84b773bfec7
I20260807 15:06:56.765329 24329 raft_consensus.cc:348] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673735580ce466ca0e443eb4a69545a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45853 } }
I20260807 15:06:56.765486 24329 raft_consensus.cc:374] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:56.765527 24329 raft_consensus.cc:729] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b673735580ce466ca0e443eb4a69545a, State: Initialized, Role: FOLLOWER
I20260807 15:06:56.765661 24329 consensus_queue.cc:260] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673735580ce466ca0e443eb4a69545a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45853 } }
I20260807 15:06:56.765745 24329 raft_consensus.cc:388] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:56.765782 24329 raft_consensus.cc:482] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:56.765829 24329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:56.766462 24329 raft_consensus.cc:504] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673735580ce466ca0e443eb4a69545a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45853 } }
I20260807 15:06:56.766559 24329 leader_election.cc:302] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b673735580ce466ca0e443eb4a69545a; no voters:
I20260807 15:06:56.766709 24329 leader_election.cc:288] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:56.766758 24335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:56.766966 24335 raft_consensus.cc:686] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [term 1 LEADER]: Becoming Leader. State: Replica: b673735580ce466ca0e443eb4a69545a, State: Running, Role: LEADER
I20260807 15:06:56.767002 24329 sys_catalog.cc:564] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:56.767117 24335 consensus_queue.cc:237] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673735580ce466ca0e443eb4a69545a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45853 } }
I20260807 15:06:56.767632 24338 sys_catalog.cc:455] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b673735580ce466ca0e443eb4a69545a. Latest consensus state: current_term: 1 leader_uuid: "b673735580ce466ca0e443eb4a69545a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673735580ce466ca0e443eb4a69545a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45853 } } }
I20260807 15:06:56.767639 24337 sys_catalog.cc:455] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b673735580ce466ca0e443eb4a69545a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b673735580ce466ca0e443eb4a69545a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45853 } } }
I20260807 15:06:56.767756 24338 sys_catalog.cc:458] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:56.767760 24337 sys_catalog.cc:458] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:56.768013 24343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:56.768543 24343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:56.769929 24343 catalog_manager.cc:1347] Generated new cluster ID: fab0eb9a93a745409301e306ab315288
I20260807 15:06:56.769975 24343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:06:56 dist-test-slave-5bn2 krb5kdc[24204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:56.775161 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:56.764416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56220 (local address 127.18.129.62:45853)
0807 15:06:56.764585 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:56.764595 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:56.764797 (+ 202us) server_negotiation.cc:407] Connection header received
0807 15:06:56.765294 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:56.765300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:56.765343 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:56.765450 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:56.766253 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:56.767447 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:56.768465 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:56.768944 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:56.769023 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:56.771737 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:56.771741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:56.771742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:56.771753 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:56.773564 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:56.773954 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:56.773956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:56.773957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:56.774065 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:56.774274 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:56.774275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:56.774275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:56.774548 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:56.774698 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:56.774802 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 15:06:56.774967 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260807 15:06:56.776260 24343 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:56.776716 24343 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:56.782146 24343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b673735580ce466ca0e443eb4a69545a: Generated new TSK 0
I20260807 15:06:56.782294 24343 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:06:56.809278 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45853
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:42769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:56.891070 24365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:56.891268 24365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:56.891299 24365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:56.892429 24365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:56.892468 24365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:56.892503 24365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:56.894037 24365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:42769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:56.894266 24365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:56.894475 24365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:56.899494 24375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:56.899504 24371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:56.899499 24370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:56.899786 24365 server_base.cc:1034] running on GCE node
I20260807 15:06:56.900013 24365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:56.900247 24365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:56.901399 24365 hybrid_clock.cc:648] HybridClock initialized: now 1786115216901358 us; error 39 us; skew 500 ppm
Aug 07 15:06:56 dist-test-slave-5bn2 krb5kdc[24204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115216, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:56.903903 24365 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:06:56.904335 24365 webserver.cc:460] Webserver started at http://127.18.129.1:39117/ using document root <none> and password file <none>
I20260807 15:06:56.904561 24365 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:56.904634 24365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:56.904762 24365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:56.905802 24365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "ddd68aaa790f417aabe0a2c2c3f8075e"
format_stamp: "Formatted at 2026-08-07 15:06:56 on dist-test-slave-5bn2"
server_key: "0050d61bd1b5af5db98ccaff93fdc0ff"
server_key_iv: "3e1325c3be207b7a7569aa25a00e45c2"
server_key_version: "encryptionkey@0"
I20260807 15:06:56.906129 24365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "ddd68aaa790f417aabe0a2c2c3f8075e"
format_stamp: "Formatted at 2026-08-07 15:06:56 on dist-test-slave-5bn2"
server_key: "0050d61bd1b5af5db98ccaff93fdc0ff"
server_key_iv: "3e1325c3be207b7a7569aa25a00e45c2"
server_key_version: "encryptionkey@0"
I20260807 15:06:56.907421 24365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 15:06:56.908107 24385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:56.908252 24365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:56.908339 24365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "ddd68aaa790f417aabe0a2c2c3f8075e"
format_stamp: "Formatted at 2026-08-07 15:06:56 on dist-test-slave-5bn2"
server_key: "0050d61bd1b5af5db98ccaff93fdc0ff"
server_key_iv: "3e1325c3be207b7a7569aa25a00e45c2"
server_key_version: "encryptionkey@0"
I20260807 15:06:56.908398 24365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:56.917341 24365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:56.919449 24365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:56.919610 24365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:56.919819 24365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:56.920104 24365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:56.920138 24365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:56.920166 24365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:56.920183 24365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:56.925601 24365 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:41005
I20260807 15:06:56.925634 24551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:41005 every 8 connection(s)
I20260807 15:06:56.925937 24365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:06:56 dist-test-slave-5bn2 krb5kdc[24204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115216, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:56.934635 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:56.926596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:33641 (local address 127.18.129.62:45853)
0807 15:06:56.926642 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:56.926644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:56.926879 (+ 235us) server_negotiation.cc:407] Connection header received
0807 15:06:56.927865 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:56.927866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:56.927883 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:56.927937 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:56.928563 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:56.929056 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:56.930029 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:56.930248 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:56.930305 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:56.932345 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:56.932347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:56.932347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:56.932356 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:56.933617 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:56.933935 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:56.933935 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:56.933936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:56.933981 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:56.934162 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:56.934162 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:56.934162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:56.934254 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:56.934296 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:56.934502 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 15:06:56.934568 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:56.934664 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 24365
I20260807 15:06:56.934764 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:56.934939 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:56.926563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45853 (local address 127.18.129.1:33641)
0807 15:06:56.926750 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:56.926785 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:56.927757 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:56.928019 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:56.928023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:56.928284 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:56.928469 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:56.928474 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:56.929143 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:56.929144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:56.929945 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:56.929947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:56.930321 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:56.930322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:56.930446 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:56.930902 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:56.930910 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:56.932248 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:56.933681 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:56.933685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:56.933686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:56.933882 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:56.934023 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:56.934024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:56.934024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:56.934119 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:56.934295 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:56.934298 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:56.934402 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 15:06:56.934480 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 15:06:56.934676 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260807 15:06:56.935001 18948 external_mini_cluster.cc:1428] Setting key 2a7afc31fb9f857793a6e0d5b9d7ead5
I20260807 15:06:56.935514 24552 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45853
I20260807 15:06:56.935609 24552 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:56.935801 24552 heartbeater.cc:511] Master 127.18.129.62:45853 requested a full tablet report, sending...
I20260807 15:06:56.936328 24271 ts_manager.cc:194] Registered new tserver with Master: ddd68aaa790f417aabe0a2c2c3f8075e (127.18.129.1:41005)
I20260807 15:06:56.936913 24271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:33641
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:06:56.959359 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45853
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:42769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:57.039958 24557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:57.040174 24557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:57.040208 24557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:57.041430 24557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:57.041474 24557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:57.041509 24557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:57.043028 24557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:42769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:57.043273 24557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:57.043486 24557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:57.048349 24564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:57.048497 24557 server_base.cc:1034] running on GCE node
W20260807 15:06:57.048310 24570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:57.048311 24565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:57.048803 24557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:57.049022 24557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:57.050168 24557 hybrid_clock.cc:648] HybridClock initialized: now 1786115217050155 us; error 27 us; skew 500 ppm
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115217, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:57.052802 24557 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:06:57.053272 24557 webserver.cc:460] Webserver started at http://127.18.129.2:39605/ using document root <none> and password file <none>
I20260807 15:06:57.053490 24557 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:57.053541 24557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:57.053668 24557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:57.054710 24557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "fd7648db7dba4db8ae0c0e3258d2ccbd"
format_stamp: "Formatted at 2026-08-07 15:06:57 on dist-test-slave-5bn2"
server_key: "29255654d438f6e4c6ea94ae163f70fa"
server_key_iv: "751594e0367ba1c21d40a9c1e5ccdcd9"
server_key_version: "encryptionkey@0"
I20260807 15:06:57.055107 24557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "fd7648db7dba4db8ae0c0e3258d2ccbd"
format_stamp: "Formatted at 2026-08-07 15:06:57 on dist-test-slave-5bn2"
server_key: "29255654d438f6e4c6ea94ae163f70fa"
server_key_iv: "751594e0367ba1c21d40a9c1e5ccdcd9"
server_key_version: "encryptionkey@0"
I20260807 15:06:57.056341 24557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:57.057129 24578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:57.057286 24557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:57.057370 24557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "fd7648db7dba4db8ae0c0e3258d2ccbd"
format_stamp: "Formatted at 2026-08-07 15:06:57 on dist-test-slave-5bn2"
server_key: "29255654d438f6e4c6ea94ae163f70fa"
server_key_iv: "751594e0367ba1c21d40a9c1e5ccdcd9"
server_key_version: "encryptionkey@0"
I20260807 15:06:57.057439 24557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:57.074288 24557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:57.076174 24557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:57.076298 24557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:57.076515 24557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:57.076812 24557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:57.076866 24557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:57.076892 24557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:57.076905 24557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:57.082402 24557 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43077
I20260807 15:06:57.082446 24746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43077 every 8 connection(s)
I20260807 15:06:57.082738 24557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:06:57.083694 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 24557
I20260807 15:06:57.083793 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:57.083956 18948 external_mini_cluster.cc:1428] Setting key 030f7c7efe12dcceecc0be843c155ad0
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115217, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
I20260807 15:06:57.093009 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:57.083442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:33133 (local address 127.18.129.62:45853)
0807 15:06:57.083505 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:57.083507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:57.083687 (+ 180us) server_negotiation.cc:407] Connection header received
0807 15:06:57.084792 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:57.084794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:57.084807 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:57.084928 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:57.085598 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.086275 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.087682 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.087940 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.088012 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:57.090406 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:57.090407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:57.090408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:57.090415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:57.091716 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:57.092123 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:57.092124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:57.092124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:57.092178 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:57.092406 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:57.092407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:57.092407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:57.092516 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:57.092572 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:57.092826 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 15:06:57.092918 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:06:57.093166 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:57.083401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45853 (local address 127.18.129.2:33133)
0807 15:06:57.083559 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:57.083603 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:57.084678 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:57.084984 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:57.084987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:57.085320 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:57.085510 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:57.085515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.086371 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.086372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:57.087593 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:57.087596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.088088 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.088089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:57.088252 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:57.088809 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:57.088819 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:57.090307 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:57.091792 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:57.091799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:57.091801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:57.092054 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:57.092233 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:57.092234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:57.092234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:57.092347 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:57.092572 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:57.092579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:57.092709 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:06:57.092796 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:06:57.092956 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":101,"threads_started":1}
I20260807 15:06:57.093636 24747 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45853
I20260807 15:06:57.093729 24747 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:57.093914 24747 heartbeater.cc:511] Master 127.18.129.62:45853 requested a full tablet report, sending...
I20260807 15:06:57.094280 24271 ts_manager.cc:194] Registered new tserver with Master: fd7648db7dba4db8ae0c0e3258d2ccbd (127.18.129.2:43077)
I20260807 15:06:57.094631 24271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:33133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:06:57.109119 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45853
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:42769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:57.189136 24755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:57.189334 24755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:57.189355 24755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:57.190493 24755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:57.190536 24755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:57.190564 24755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:57.192085 24755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:42769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:57.192296 24755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:57.192497 24755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:57.197479 24767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:57.197479 24761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:57.197503 24762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:57.197724 24755 server_base.cc:1034] running on GCE node
I20260807 15:06:57.197925 24755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:57.198143 24755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:57.199278 24755 hybrid_clock.cc:648] HybridClock initialized: now 1786115217199262 us; error 25 us; skew 500 ppm
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115217, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:57.201687 24755 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:06:57.202100 24755 webserver.cc:460] Webserver started at http://127.18.129.3:40529/ using document root <none> and password file <none>
I20260807 15:06:57.202288 24755 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:57.202334 24755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:57.202459 24755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:57.203490 24755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "95e7b61324d043fb8941e46bd21553da"
format_stamp: "Formatted at 2026-08-07 15:06:57 on dist-test-slave-5bn2"
server_key: "96446dab0507f80afa95268e9d1e8f50"
server_key_iv: "7520ca9d6c55f757966df9d1a3dfc8db"
server_key_version: "encryptionkey@0"
I20260807 15:06:57.203858 24755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "95e7b61324d043fb8941e46bd21553da"
format_stamp: "Formatted at 2026-08-07 15:06:57 on dist-test-slave-5bn2"
server_key: "96446dab0507f80afa95268e9d1e8f50"
server_key_iv: "7520ca9d6c55f757966df9d1a3dfc8db"
server_key_version: "encryptionkey@0"
I20260807 15:06:57.205089 24755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:06:57.205735 24777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:57.205880 24755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:57.205976 24755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "95e7b61324d043fb8941e46bd21553da"
format_stamp: "Formatted at 2026-08-07 15:06:57 on dist-test-slave-5bn2"
server_key: "96446dab0507f80afa95268e9d1e8f50"
server_key_iv: "7520ca9d6c55f757966df9d1a3dfc8db"
server_key_version: "encryptionkey@0"
I20260807 15:06:57.206038 24755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:57.224251 24755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:57.226111 24755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:57.226244 24755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:57.226464 24755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:57.226773 24755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:57.226807 24755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:57.226836 24755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:57.226886 24755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:57.232507 24755 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:34977
I20260807 15:06:57.232568 24943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:34977 every 8 connection(s)
I20260807 15:06:57.232882 24755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:06:57.233973 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 24755
I20260807 15:06:57.234078 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:57.234252 18948 external_mini_cluster.cc:1428] Setting key bc6e47812f2dd220d0bf0ca4b734a57a
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115217, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:57.241853 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:57.233544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:41107 (local address 127.18.129.62:45853)
0807 15:06:57.233591 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:57.233593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:57.233798 (+ 205us) server_negotiation.cc:407] Connection header received
0807 15:06:57.234899 (+ 1101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:57.234900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:57.234913 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:57.234959 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:57.235625 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.236113 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.237071 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.237267 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.237321 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:57.239373 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:57.239374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:57.239375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:57.239384 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:57.240525 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:57.240961 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:57.240962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:57.240962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:57.241019 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:57.241235 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:57.241236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:57.241236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:57.241346 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:57.241411 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:57.241681 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 15:06:57.241761 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:06:57.242030 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:57.233504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45853 (local address 127.18.129.3:41107)
0807 15:06:57.233685 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:57.233722 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:57.234800 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:57.235045 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:57.235050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:57.235347 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:57.235537 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:57.235543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.236194 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.236196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:57.236989 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:57.236992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.237337 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.237338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:57.237469 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:57.237898 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:57.237906 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:57.239268 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:57.240607 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:57.240615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:57.240617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:57.240885 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:57.241074 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:57.241075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:57.241075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:57.241178 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:57.241410 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:57.241418 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:57.241561 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 15:06:57.241648 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:06:57.241799 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260807 15:06:57.242447 24944 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45853
I20260807 15:06:57.242539 24944 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:57.242712 24944 heartbeater.cc:511] Master 127.18.129.62:45853 requested a full tablet report, sending...
I20260807 15:06:57.243113 24271 ts_manager.cc:194] Registered new tserver with Master: 95e7b61324d043fb8941e46bd21553da (127.18.129.3:34977)
I20260807 15:06:57.243533 24271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:41107
I20260807 15:06:57.245786 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:06:57.269539 24957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:57.261574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46094 (local address 127.18.129.1:41005)
0807 15:06:57.261741 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:57.261745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:57.261754 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:06:57.261780 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:57.261782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:57.261795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:57.261858 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:57.262412 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.263243 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.263963 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.264287 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.264347 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:57.266793 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:57.266802 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:57.266802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:57.266810 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:57.268362 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:57.268747 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:57.268750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:57.268751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:57.268835 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:57.269053 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:57.269054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:57.269054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:57.269201 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:57.269333 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:57.269369 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 15:06:57.269433 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:57.276567 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:57.270772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.18.129.62:45853)
0807 15:06:57.270815 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:57.270817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:57.270827 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:06:57.270872 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:57.270873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:57.270883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:57.270934 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:57.271336 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.271800 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.272384 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:57.272551 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:57.272590 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:57.274422 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:57.274424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:57.274424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:57.274431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:57.275712 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:57.275984 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:57.275984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:57.275985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:57.276028 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:57.276165 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:57.276165 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:57.276165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:57.276294 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:57.276361 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:57.276427 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 15:06:57.276485 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 15:06:57.276809 24271 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56228
I20260807 15:06:57.277130 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 24365
I20260807 15:06:57.289268 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 24557
I20260807 15:06:57.298192 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 24755
I20260807 15:06:57.324910 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 24222
2026-08-07T15:06:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1974 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24963](info): set up 2 sockets
Aug 07 15:06:57 dist-test-slave-5bn2 krb5kdc[24963](info): commencing operation
krb5kdc: starting...
W20260807 15:06:58.460638 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:06:58 dist-test-slave-5bn2 krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:06:58.575809 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37747
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:37747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:58.655267 24978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:58.655469 24978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:58.655490 24978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:58.656575 24978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:58.656608 24978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:58.656621 24978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:58.656632 24978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:58.656648 24978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:58.658075 24978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:37747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:58.658264 24978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:58.658432 24978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:58.663110 24987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:58.663170 24989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:58.663168 24986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:58.663204 24978 server_base.cc:1034] running on GCE node
I20260807 15:06:58.663529 24978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:58.663789 24978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:58.664945 24978 hybrid_clock.cc:648] HybridClock initialized: now 1786115218664925 us; error 35 us; skew 500 ppm
Aug 07 15:06:58 dist-test-slave-5bn2 krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115218, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:58.667371 24978 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:06:58.667692 24978 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:58.667740 24978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:58.667858 24978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:58.668886 24978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "08c3c29a398b4f02b09a45757e04affc"
format_stamp: "Formatted at 2026-08-07 15:06:58 on dist-test-slave-5bn2"
server_key: "0c82bc0071491b1a2ca8d52d347438c4"
server_key_iv: "b76848075ce0502d1dde711336766737"
server_key_version: "encryptionkey@0"
I20260807 15:06:58.669214 24978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "08c3c29a398b4f02b09a45757e04affc"
format_stamp: "Formatted at 2026-08-07 15:06:58 on dist-test-slave-5bn2"
server_key: "0c82bc0071491b1a2ca8d52d347438c4"
server_key_iv: "b76848075ce0502d1dde711336766737"
server_key_version: "encryptionkey@0"
I20260807 15:06:58.670491 24978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:06:58.671191 24997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:58.671335 24978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:58.671417 24978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "08c3c29a398b4f02b09a45757e04affc"
format_stamp: "Formatted at 2026-08-07 15:06:58 on dist-test-slave-5bn2"
server_key: "0c82bc0071491b1a2ca8d52d347438c4"
server_key_iv: "b76848075ce0502d1dde711336766737"
server_key_version: "encryptionkey@0"
I20260807 15:06:58.671482 24978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:58.685082 24978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:58.687182 24978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:58.687310 24978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:58.691179 24978 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:37747
I20260807 15:06:58.691207 25085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:37747 every 8 connection(s)
I20260807 15:06:58.691458 24978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:58.692556 25086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:58.694819 25086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc: Bootstrap starting.
I20260807 15:06:58.695420 25086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:58.695654 25086 log.cc:824] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:58.696276 25086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc: No bootstrap required, opened a new log
I20260807 15:06:58.697796 25086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c3c29a398b4f02b09a45757e04affc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37747 } }
I20260807 15:06:58.697935 25086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:58.697962 25086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c3c29a398b4f02b09a45757e04affc, State: Initialized, Role: FOLLOWER
I20260807 15:06:58.698083 25086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c3c29a398b4f02b09a45757e04affc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37747 } }
I20260807 15:06:58.698159 25086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:58.698189 25086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:58.698259 25086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:58.699120 25086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c3c29a398b4f02b09a45757e04affc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37747 } }
I20260807 15:06:58.699216 25086 leader_election.cc:302] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08c3c29a398b4f02b09a45757e04affc; no voters:
I20260807 15:06:58.699353 25086 leader_election.cc:288] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:58.699425 25089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:58.699565 25089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [term 1 LEADER]: Becoming Leader. State: Replica: 08c3c29a398b4f02b09a45757e04affc, State: Running, Role: LEADER
I20260807 15:06:58.699622 25086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:58.699649 25089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c3c29a398b4f02b09a45757e04affc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37747 } }
I20260807 15:06:58.699999 25091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08c3c29a398b4f02b09a45757e04affc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c3c29a398b4f02b09a45757e04affc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37747 } } }
I20260807 15:06:58.700027 25092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08c3c29a398b4f02b09a45757e04affc. Latest consensus state: current_term: 1 leader_uuid: "08c3c29a398b4f02b09a45757e04affc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c3c29a398b4f02b09a45757e04affc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37747 } } }
I20260807 15:06:58.700116 25091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:58.700132 25092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:58.700342 25096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:58.700100 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 24978
I20260807 15:06:58.700584 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:58.700863 25096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:58.700929 18948 external_mini_cluster.cc:1428] Setting key 26a8962a5b6331300682ff071e5e12ee
I20260807 15:06:58.702678 25096 catalog_manager.cc:1347] Generated new cluster ID: 2f791a5293f2483087eec4eb2c48d5af
I20260807 15:06:58.702735 25096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:06:58 dist-test-slave-5bn2 krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:58.710783 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:58.701376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46018 (local address 127.18.129.62:37747)
0807 15:06:58.701573 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:58.701583 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:58.701987 (+ 404us) server_negotiation.cc:407] Connection header received
0807 15:06:58.702128 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:58.702134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:58.702180 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:58.702308 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:58.703055 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:58.703973 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:58.704699 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:58.705136 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:58.705229 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:58.707410 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:58.707415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:58.707415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:58.707427 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:58.709313 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:58.709632 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:58.709635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:58.709635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:58.709743 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:58.709919 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:58.709920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:58.709920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:58.710169 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:58.710319 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:58.710397 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:06:58.710621 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260807 15:06:58.713203 25096 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:58.713680 25096 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:58.722234 25096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc: Generated new TSK 0
I20260807 15:06:58.722394 25096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:06:58.754961 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37747
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:58.846513 25124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:58.846729 25124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:58.846752 25124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:58.847921 25124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:58.847965 25124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:58.847991 25124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:58.849478 25124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:58.849695 25124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:58.849887 25124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:58.854735 25134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:58.854749 25130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:58.854811 25131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:58.854955 25124 server_base.cc:1034] running on GCE node
I20260807 15:06:58.855316 25124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:58.855506 25124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:58.856645 25124 hybrid_clock.cc:648] HybridClock initialized: now 1786115218856632 us; error 29 us; skew 500 ppm
Aug 07 15:06:58 dist-test-slave-5bn2 krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115218, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:58.859041 25124 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:06:58.859352 25124 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:58.859403 25124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:58.859526 25124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:58.860630 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "ede1fb2929084d82bed7f84307fd4644"
format_stamp: "Formatted at 2026-08-07 15:06:58 on dist-test-slave-5bn2"
server_key: "a7b099da680acbaf0f0b4ed1f2278f52"
server_key_iv: "77ac002f8fce704462562830397832b7"
server_key_version: "encryptionkey@0"
I20260807 15:06:58.860970 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "ede1fb2929084d82bed7f84307fd4644"
format_stamp: "Formatted at 2026-08-07 15:06:58 on dist-test-slave-5bn2"
server_key: "a7b099da680acbaf0f0b4ed1f2278f52"
server_key_iv: "77ac002f8fce704462562830397832b7"
server_key_version: "encryptionkey@0"
I20260807 15:06:58.862181 25124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:58.862821 25143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:58.862993 25124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:58.863078 25124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "ede1fb2929084d82bed7f84307fd4644"
format_stamp: "Formatted at 2026-08-07 15:06:58 on dist-test-slave-5bn2"
server_key: "a7b099da680acbaf0f0b4ed1f2278f52"
server_key_iv: "77ac002f8fce704462562830397832b7"
server_key_version: "encryptionkey@0"
I20260807 15:06:58.863142 25124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:58.889144 25124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:58.891064 25124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:58.891191 25124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:58.891383 25124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:58.891698 25124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:58.891736 25124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:58.891767 25124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:58.891784 25124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:58.897321 25124 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:36269
I20260807 15:06:58.897399 25302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:36269 every 8 connection(s)
I20260807 15:06:58.897647 25124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:06:58.900380 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 25124
I20260807 15:06:58.900477 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:58.900692 18948 external_mini_cluster.cc:1428] Setting key 8d9ab3f04220e185252164fbd80da578
Aug 07 15:06:58 dist-test-slave-5bn2 krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115218, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:58.907729 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:58.898328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:37001 (local address 127.18.129.62:37747)
0807 15:06:58.898401 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:58.898403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:58.898600 (+ 197us) server_negotiation.cc:407] Connection header received
0807 15:06:58.899193 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:58.899195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:58.899208 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:58.899259 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:58.900262 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:58.900917 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:58.902191 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:58.902396 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:58.902451 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:58.904803 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:58.904805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:58.904805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:58.904811 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:58.906294 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:58.906677 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:58.906678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:58.906679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:58.906746 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:58.907012 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:58.907014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:58.907014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:58.907184 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:58.907261 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:58.907541 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 15:06:58.907635 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:06:58.907755 25307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:58.898313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37747 (local address 127.18.129.1:37001)
0807 15:06:58.898471 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:58.898511 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:58.899086 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:58.899330 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:58.899334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:58.899647 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:58.900155 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:58.900161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:58.900995 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:58.900997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:58.902107 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:58.902110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:58.902472 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:58.902473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:58.902609 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:58.903133 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:58.903143 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:58.904715 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:58.906372 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:58.906380 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:58.906382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:58.906604 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:58.906806 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:58.906806 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:58.906807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:58.906936 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:58.907253 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:58.907257 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:58.907381 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 15:06:58.907457 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:06:58.907587 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260807 15:06:58.908234 25303 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37747
I20260807 15:06:58.908324 25303 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:58.908465 25303 heartbeater.cc:511] Master 127.18.129.62:37747 requested a full tablet report, sending...
I20260807 15:06:58.908942 25025 ts_manager.cc:194] Registered new tserver with Master: ede1fb2929084d82bed7f84307fd4644 (127.18.129.1:36269)
I20260807 15:06:58.909515 25025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:37001
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:06:58.925818 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37747
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:59.006177 25310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:59.006386 25310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:59.006417 25310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:59.007586 25310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:59.007627 25310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:59.007663 25310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:59.009173 25310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:59.009429 25310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:59.009644 25310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:59.014642 25320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.014659 25318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:59.014731 25310 server_base.cc:1034] running on GCE node
W20260807 15:06:59.014760 25322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:59.015133 25310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:59.015352 25310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:59.016490 25310 hybrid_clock.cc:648] HybridClock initialized: now 1786115219016471 us; error 30 us; skew 500 ppm
Aug 07 15:06:59 dist-test-slave-5bn2 krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115219, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:59.018939 25310 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:06:59.019265 25310 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:59.019318 25310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:59.019444 25310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:59.020426 25310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "10036d0ea6c24a34bd635c972ffa135c"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "37cddab8ed3451fb4bdd2ddc3efe526e"
server_key_iv: "864fa6ae117c971e42ab9ccd046b254c"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.020763 25310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "10036d0ea6c24a34bd635c972ffa135c"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "37cddab8ed3451fb4bdd2ddc3efe526e"
server_key_iv: "864fa6ae117c971e42ab9ccd046b254c"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.021979 25310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:59.022612 25332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.022763 25310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:59.022871 25310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "10036d0ea6c24a34bd635c972ffa135c"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "37cddab8ed3451fb4bdd2ddc3efe526e"
server_key_iv: "864fa6ae117c971e42ab9ccd046b254c"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.022935 25310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:59.038311 25310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:59.039973 25310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:59.040113 25310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:59.040311 25310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:59.040598 25310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:59.040633 25310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.040660 25310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:59.040685 25310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.046478 25310 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:35403
I20260807 15:06:59.046511 25497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:35403 every 8 connection(s)
I20260807 15:06:59.046763 25310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:06:59.050371 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 25310
I20260807 15:06:59.050479 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:59.050674 18948 external_mini_cluster.cc:1428] Setting key 1de7f092c71e7bd161f707f614d47844
Aug 07 15:06:59 dist-test-slave-5bn2 krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115219, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:59.056046 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.047475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:53975 (local address 127.18.129.62:37747)
0807 15:06:59.047522 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.047524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.047713 (+ 189us) server_negotiation.cc:407] Connection header received
0807 15:06:59.048173 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.048173 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.048182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.048235 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:59.049091 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.049587 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.050761 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.050994 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.051080 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.053326 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:59.053328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:59.053328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:59.053336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:59.054713 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:59.055166 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:59.055167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:59.055167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:59.055221 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:59.055466 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:59.055466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:59.055466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:59.055567 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:59.055613 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.055893 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.055970 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:06:59.056305 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.047454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37747 (local address 127.18.129.2:53975)
0807 15:06:59.047603 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.047645 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.048098 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.048288 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.048291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.048538 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:59.049020 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.049025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.049648 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.049648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.050673 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.050676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.051096 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.051098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.051204 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.051685 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:59.051694 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:59.053218 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:59.054789 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:59.054796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:59.054798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:59.055100 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:59.055276 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:59.055277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:59.055277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:59.055410 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:59.055616 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:59.055622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:59.055773 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.055889 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.056078 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260807 15:06:59.056847 25498 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37747
I20260807 15:06:59.056951 25498 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:59.057113 25498 heartbeater.cc:511] Master 127.18.129.62:37747 requested a full tablet report, sending...
I20260807 15:06:59.057521 25025 ts_manager.cc:194] Registered new tserver with Master: 10036d0ea6c24a34bd635c972ffa135c (127.18.129.2:35403)
I20260807 15:06:59.057890 25025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:53975
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:06:59.077066 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37747
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:06:59.156886 25506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:59.157083 25506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:59.157115 25506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:59.158239 25506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:59.158279 25506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:59.158314 25506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:59.159812 25506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:59.160048 25506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:59.160249 25506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:59.165108 25523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.165148 25519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.165196 25514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:59.165230 25506 server_base.cc:1034] running on GCE node
I20260807 15:06:59.165470 25506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:59.165699 25506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:59.166827 25506 hybrid_clock.cc:648] HybridClock initialized: now 1786115219166813 us; error 30 us; skew 500 ppm
Aug 07 15:06:59 dist-test-slave-5bn2 krb5kdc[24963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115219, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:06:59.169201 25506 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:06:59.169523 25506 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:59.169577 25506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:59.169699 25506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:59.170725 25506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "512f638f4a728fb3e7d79ebe273d8fae"
server_key_iv: "0f99a5dea6a93a2bfcaf863cb738868c"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.171080 25506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "512f638f4a728fb3e7d79ebe273d8fae"
server_key_iv: "0f99a5dea6a93a2bfcaf863cb738868c"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.172302 25506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:06:59.172962 25534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.173110 25506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:59.173195 25506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "512f638f4a728fb3e7d79ebe273d8fae"
server_key_iv: "0f99a5dea6a93a2bfcaf863cb738868c"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.173256 25506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:59.187196 25506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:59.189054 25506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:59.189188 25506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:59.189380 25506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:59.189674 25506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:59.189708 25506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.189735 25506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:59.189761 25506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.195309 25506 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:41727
I20260807 15:06:59.195377 25704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:41727 every 8 connection(s)
I20260807 15:06:59.195600 25506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:06:59 dist-test-slave-5bn2 krb5kdc[24963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115219, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:06:59.201768 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 25506
I20260807 15:06:59.201864 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:06:59.202061 18948 external_mini_cluster.cc:1428] Setting key 7b0549a56058a599cdfdb4940d17a584
I20260807 15:06:59.204474 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.196261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:41843 (local address 127.18.129.62:37747)
0807 15:06:59.196336 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.196339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.196511 (+ 172us) server_negotiation.cc:407] Connection header received
0807 15:06:59.197045 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.197047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.197057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.197106 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:59.197982 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.198473 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.199634 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.199819 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.199867 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.202015 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:59.202017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:59.202017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:59.202026 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:59.203468 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:59.203804 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:59.203805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:59.203805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:59.203853 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:59.204014 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:59.204014 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:59.204015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:59.204112 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:59.204158 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.204349 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.204410 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:06:59.204638 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.196240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37747 (local address 127.18.129.3:41843)
0807 15:06:59.196397 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.196433 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.196939 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.197163 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.197166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.197415 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:59.197913 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.197918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.198574 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.198575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.199561 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.199565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.199923 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.199925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.200035 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.200490 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:06:59.200499 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:59.201937 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:59.203533 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:59.203539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:59.203541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:59.203747 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:59.203892 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:06:59.203892 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:06:59.203893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:06:59.203975 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:06:59.204154 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:59.204156 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:59.204259 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.204332 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.204444 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260807 15:06:59.205037 25707 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37747
I20260807 15:06:59.205118 25707 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:59.205252 25707 heartbeater.cc:511] Master 127.18.129.62:37747 requested a full tablet report, sending...
I20260807 15:06:59.205575 25025 ts_manager.cc:194] Registered new tserver with Master: 7e7a54a1c37545fd8b68c052e7ca1d5d (127.18.129.3:41727)
I20260807 15:06:59.205922 25025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:41843
I20260807 15:06:59.213824 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:06:59.220427 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.215011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46034 (local address 127.18.129.62:37747)
0807 15:06:59.215066 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.215067 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.215159 (+ 92us) server_negotiation.cc:407] Connection header received
0807 15:06:59.215218 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.215219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.215229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.215281 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:59.215792 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.216258 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.216933 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.217137 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.217190 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.218207 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:59.218208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:59.218208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:06:59.218215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:59.219385 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:59.219698 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:59.219699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:59.219700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:59.219752 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:06:59.219911 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:06:59.219912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:06:59.219912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:06:59.220056 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:59.220145 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.220267 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.220339 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:06:59.221903 25025 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:06:59.223882 25025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:06:59.230060 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.226236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.18.129.2:35403)
0807 15:06:59.226503 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.226507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.226519 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:06:59.226645 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.226648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.226665 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.226732 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:59.227036 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.227785 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.228964 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.229694 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.229750 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.229812 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.229941 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.229977 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
I20260807 15:06:59.230051 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.226052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:35403 (local address 127.0.0.1:50132)
0807 15:06:59.226433 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.226474 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.226533 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.226740 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.226743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.226746 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:59.226927 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.226929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.227884 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.227885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.228860 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.228863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.229767 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.229768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.229844 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.229857 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.229920 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.229974 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260807 15:06:59.230366 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.226059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:41727 (local address 127.0.0.1:43468)
0807 15:06:59.226681 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.226688 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.226726 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.227093 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.227094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.227097 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:59.227144 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.227146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.228453 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.228454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.229314 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.229317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.230157 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.230158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.230220 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.230225 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.230255 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.230293 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260807 15:06:59.230648 25721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.226052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36269 (local address 127.0.0.1:39478)
0807 15:06:59.226556 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.226564 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.226612 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.227072 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.227073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.227075 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:59.227124 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.227126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.228440 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.228441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.229529 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.229532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.230404 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.230405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.230482 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.230486 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.230541 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.230577 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":35,"thread_start_us":56,"threads_started":1}
I20260807 15:06:59.230703 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.226243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43468 (local address 127.18.129.3:41727)
0807 15:06:59.226455 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.226461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.226799 (+ 338us) server_negotiation.cc:407] Connection header received
0807 15:06:59.226870 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.226872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.226889 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.226967 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:59.227225 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.228330 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.229391 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.230023 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.230480 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.230555 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.230579 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.230616 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":86,"threads_started":1}
I20260807 15:06:59.230703 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.226300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.18.129.1:36269)
0807 15:06:59.226833 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.226837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.226866 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:06:59.226901 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.226903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.226918 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.227012 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:59.227196 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.228325 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.229613 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.230317 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.230379 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.230551 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.230580 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.230616 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":78,"threads_started":1}
I20260807 15:06:59.231371 25599 tablet_service.cc:1467] Processing CreateTablet for tablet e637d4e8a6a849b5857a27a541f42f81 (DEFAULT_TABLE table=test-table [id=6375ea3ba64a468e93ea58cfd892b37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:59.231334 25404 tablet_service.cc:1467] Processing CreateTablet for tablet e637d4e8a6a849b5857a27a541f42f81 (DEFAULT_TABLE table=test-table [id=6375ea3ba64a468e93ea58cfd892b37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:59.231523 25212 tablet_service.cc:1467] Processing CreateTablet for tablet e637d4e8a6a849b5857a27a541f42f81 (DEFAULT_TABLE table=test-table [id=6375ea3ba64a468e93ea58cfd892b37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:06:59.231818 25599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e637d4e8a6a849b5857a27a541f42f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:59.231818 25404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e637d4e8a6a849b5857a27a541f42f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:59.231879 25212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e637d4e8a6a849b5857a27a541f42f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:59.234403 25728 tablet_bootstrap.cc:492] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644: Bootstrap starting.
I20260807 15:06:59.235059 25730 tablet_bootstrap.cc:492] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Bootstrap starting.
I20260807 15:06:59.235109 25728 tablet_bootstrap.cc:654] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:59.235404 25733 tablet_bootstrap.cc:492] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Bootstrap starting.
I20260807 15:06:59.235843 25730 tablet_bootstrap.cc:654] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:59.235955 25733 tablet_bootstrap.cc:654] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:59.236110 25728 log.cc:824] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:59.236560 25733 log.cc:824] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:59.236572 25730 log.cc:824] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:59.237030 25728 tablet_bootstrap.cc:492] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644: No bootstrap required, opened a new log
I20260807 15:06:59.237104 25728 ts_tablet_manager.cc:1397] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:06:59.237246 25733 tablet_bootstrap.cc:492] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: No bootstrap required, opened a new log
I20260807 15:06:59.237300 25733 ts_tablet_manager.cc:1397] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:06:59.238584 25728 raft_consensus.cc:348] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.238732 25728 raft_consensus.cc:374] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:59.238768 25728 raft_consensus.cc:729] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede1fb2929084d82bed7f84307fd4644, State: Initialized, Role: FOLLOWER
I20260807 15:06:59.238760 25733 raft_consensus.cc:348] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.238945 25733 raft_consensus.cc:374] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:59.238911 25728 consensus_queue.cc:260] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.238988 25733 raft_consensus.cc:729] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7a54a1c37545fd8b68c052e7ca1d5d, State: Initialized, Role: FOLLOWER
I20260807 15:06:59.239107 25733 consensus_queue.cc:260] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.239209 25728 ts_tablet_manager.cc:1428] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:59.239225 25303 heartbeater.cc:503] Master 127.18.129.62:37747 was elected leader, sending a full tablet report...
I20260807 15:06:59.239363 25733 ts_tablet_manager.cc:1428] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:59.239492 25707 heartbeater.cc:503] Master 127.18.129.62:37747 was elected leader, sending a full tablet report...
I20260807 15:06:59.240146 25730 tablet_bootstrap.cc:492] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: No bootstrap required, opened a new log
I20260807 15:06:59.240218 25730 ts_tablet_manager.cc:1397] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 15:06:59.241683 25730 raft_consensus.cc:348] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.241837 25730 raft_consensus.cc:374] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:59.241882 25730 raft_consensus.cc:729] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10036d0ea6c24a34bd635c972ffa135c, State: Initialized, Role: FOLLOWER
I20260807 15:06:59.241989 25730 consensus_queue.cc:260] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.242237 25730 ts_tablet_manager.cc:1428] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:06:59.242307 25498 heartbeater.cc:503] Master 127.18.129.62:37747 was elected leader, sending a full tablet report...
W20260807 15:06:59.297338 25499 tablet.cc:2369] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:06:59.398363 25304 tablet.cc:2369] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:59.416949 25739 raft_consensus.cc:482] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:06:59.417101 25739 raft_consensus.cc:504] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.417461 25739 leader_election.cc:288] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10036d0ea6c24a34bd635c972ffa135c (127.18.129.2:35403), ede1fb2929084d82bed7f84307fd4644 (127.18.129.1:36269)
I20260807 15:06:59.420311 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.417799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:50363 (local address 127.18.129.1:36269)
0807 15:06:59.417843 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.417845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.417908 (+ 63us) server_negotiation.cc:407] Connection header received
0807 15:06:59.417955 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.417955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.417964 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.418029 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:59.418165 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.418633 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.419627 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.419981 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.420041 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.420049 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.420201 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.420237 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:06:59.420323 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.417645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:35403 (local address 127.18.129.3:37463)
0807 15:06:59.417707 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.417721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.417784 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.417949 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.417950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.417953 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:59.418038 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.418040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.418696 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.418697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.419520 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.419523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.420071 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.420072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.420151 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.420154 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.420201 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.420238 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:06:59.420311 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.417685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:37463 (local address 127.18.129.2:35403)
0807 15:06:59.417730 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.417733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.417764 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:06:59.417860 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.417861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.417873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.417922 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:06:59.418114 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.418609 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.419608 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.419991 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.420056 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.420065 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.420213 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.420246 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:06:59.420339 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.417645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36269 (local address 127.18.129.3:50363)
0807 15:06:59.417845 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.417854 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.417907 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.418039 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.418039 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.418041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:06:59.418104 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.418106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.418713 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.418714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.419540 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:06:59.419542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.420064 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.420065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:06:59.420139 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.420142 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.420187 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.420226 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":14,"thread_start_us":121,"threads_started":1}
I20260807 15:06:59.420713 25436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e637d4e8a6a849b5857a27a541f42f81" candidate_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10036d0ea6c24a34bd635c972ffa135c" is_pre_election: true
I20260807 15:06:59.420723 25242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e637d4e8a6a849b5857a27a541f42f81" candidate_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede1fb2929084d82bed7f84307fd4644" is_pre_election: true
I20260807 15:06:59.420881 25242 raft_consensus.cc:2454] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e7a54a1c37545fd8b68c052e7ca1d5d in term 0.
I20260807 15:06:59.420877 25436 raft_consensus.cc:2454] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e7a54a1c37545fd8b68c052e7ca1d5d in term 0.
I20260807 15:06:59.421095 25537 leader_election.cc:302] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10036d0ea6c24a34bd635c972ffa135c, 7e7a54a1c37545fd8b68c052e7ca1d5d; no voters:
I20260807 15:06:59.421273 25739 raft_consensus.cc:2781] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:06:59.421348 25739 raft_consensus.cc:482] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:06:59.421381 25739 raft_consensus.cc:3037] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:59.422109 25739 raft_consensus.cc:504] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.422237 25739 leader_election.cc:288] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [CANDIDATE]: Term 1 election: Requested vote from peers 10036d0ea6c24a34bd635c972ffa135c (127.18.129.2:35403), ede1fb2929084d82bed7f84307fd4644 (127.18.129.1:36269)
I20260807 15:06:59.422420 25242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e637d4e8a6a849b5857a27a541f42f81" candidate_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede1fb2929084d82bed7f84307fd4644"
I20260807 15:06:59.422446 25436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e637d4e8a6a849b5857a27a541f42f81" candidate_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10036d0ea6c24a34bd635c972ffa135c"
I20260807 15:06:59.422508 25436 raft_consensus.cc:3037] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:59.422501 25242 raft_consensus.cc:3037] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:59.423264 25436 raft_consensus.cc:2454] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e7a54a1c37545fd8b68c052e7ca1d5d in term 1.
I20260807 15:06:59.423272 25242 raft_consensus.cc:2454] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e7a54a1c37545fd8b68c052e7ca1d5d in term 1.
I20260807 15:06:59.423422 25535 leader_election.cc:302] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e7a54a1c37545fd8b68c052e7ca1d5d, ede1fb2929084d82bed7f84307fd4644; no voters:
I20260807 15:06:59.423525 25739 raft_consensus.cc:2781] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:59.423689 25739 raft_consensus.cc:686] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 1 LEADER]: Becoming Leader. State: Replica: 7e7a54a1c37545fd8b68c052e7ca1d5d, State: Running, Role: LEADER
I20260807 15:06:59.423782 25739 consensus_queue.cc:237] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } }
I20260807 15:06:59.424705 25024 catalog_manager.cc:5495] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e7a54a1c37545fd8b68c052e7ca1d5d (127.18.129.3). New cstate: current_term: 1 leader_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e7a54a1c37545fd8b68c052e7ca1d5d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 41727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 } health_report { overall_health: UNKNOWN } } }
W20260807 15:06:59.446113 25708 tablet.cc:2369] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:06:59.458491 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.456249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.18.129.3:41727)
0807 15:06:59.456304 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.456306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.456314 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:06:59.456349 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.456349 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.456367 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.456426 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:06:59.456580 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.457035 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.457767 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:06:59.457948 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:06:59.458002 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:06:59.458155 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:06:59.458261 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:06:59.458331 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.458372 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.458394 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:06:59.460227 25436 raft_consensus.cc:1261] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 1 FOLLOWER]: Refusing update from remote peer 7e7a54a1c37545fd8b68c052e7ca1d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:59.460260 25242 raft_consensus.cc:1261] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 1 FOLLOWER]: Refusing update from remote peer 7e7a54a1c37545fd8b68c052e7ca1d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:06:59.460582 25739 consensus_queue.cc:1035] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede1fb2929084d82bed7f84307fd4644" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:59.460723 25743 consensus_queue.cc:1035] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "10036d0ea6c24a34bd635c972ffa135c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:06:59.462921 25748 mvcc.cc:205] Tried to move back new op lower bound from 7315927938906525696 to 7315927938760544256. Current Snapshot: MvccSnapshot[applied={T|T < 7315927938906525696}]
I20260807 15:06:59.463255 25750 mvcc.cc:205] Tried to move back new op lower bound from 7315927938906525696 to 7315927938760544256. Current Snapshot: MvccSnapshot[applied={T|T < 7315927938906525696}]
I20260807 15:06:59.464159 25752 mvcc.cc:205] Tried to move back new op lower bound from 7315927938906525696 to 7315927938760544256. Current Snapshot: MvccSnapshot[applied={T|T < 7315927938906525696 or (T in {7315927938906525696})}]
I20260807 15:06:59.465193 25024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:59.465281 25024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:06:59.465982 25024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08c3c29a398b4f02b09a45757e04affc: Sending DeleteTablet for 3 replicas of tablet e637d4e8a6a849b5857a27a541f42f81
I20260807 15:06:59.466272 25404 tablet_service.cc:1514] Processing DeleteTablet for tablet e637d4e8a6a849b5857a27a541f42f81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:59 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:50132
I20260807 15:06:59.466308 25599 tablet_service.cc:1514] Processing DeleteTablet for tablet e637d4e8a6a849b5857a27a541f42f81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:59 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:43468
I20260807 15:06:59.466586 25768 tablet_replica.cc:330] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: stopping tablet replica
I20260807 15:06:59.466673 25768 raft_consensus.cc:2229] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:06:59.466639 25212 tablet_service.cc:1514] Processing DeleteTablet for tablet e637d4e8a6a849b5857a27a541f42f81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:06:59 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:39478
I20260807 15:06:59.466737 25768 raft_consensus.cc:2258] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:59.466881 25769 tablet_replica.cc:330] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644: stopping tablet replica
I20260807 15:06:59.466975 25769 raft_consensus.cc:2229] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:59.467015 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 25124
I20260807 15:06:59.467093 25769 raft_consensus.cc:2258] T e637d4e8a6a849b5857a27a541f42f81 P ede1fb2929084d82bed7f84307fd4644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:59.467515 25768 ts_tablet_manager.cc:1905] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:59.469065 25768 ts_tablet_manager.cc:1918] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:59.469126 25768 log.cc:1196] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/e637d4e8a6a849b5857a27a541f42f81
I20260807 15:06:59.469305 25768 ts_tablet_manager.cc:1939] T e637d4e8a6a849b5857a27a541f42f81 P 7e7a54a1c37545fd8b68c052e7ca1d5d: Deleting consensus metadata
I20260807 15:06:59.469624 25005 catalog_manager.cc:4841] TS 7e7a54a1c37545fd8b68c052e7ca1d5d (127.18.129.3:41727): tablet e637d4e8a6a849b5857a27a541f42f81 (table test-table [id=6375ea3ba64a468e93ea58cfd892b37f]) successfully deleted
I20260807 15:06:59.470957 25767 tablet_replica.cc:330] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: stopping tablet replica
I20260807 15:06:59.471037 25767 raft_consensus.cc:2229] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:06:59.471102 25767 raft_consensus.cc:2258] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:06:59.471455 25767 ts_tablet_manager.cc:1905] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:06:59.473129 25767 ts_tablet_manager.cc:1918] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:06:59.473181 25767 log.cc:1196] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/e637d4e8a6a849b5857a27a541f42f81
I20260807 15:06:59.473397 25767 ts_tablet_manager.cc:1939] T e637d4e8a6a849b5857a27a541f42f81 P 10036d0ea6c24a34bd635c972ffa135c: Deleting consensus metadata
I20260807 15:06:59.473737 25003 catalog_manager.cc:4841] TS 10036d0ea6c24a34bd635c972ffa135c (127.18.129.2:35403): tablet e637d4e8a6a849b5857a27a541f42f81 (table test-table [id=6375ea3ba64a468e93ea58cfd892b37f]) successfully deleted
W20260807 15:06:59.486764 25001 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:36269 (error 108)
I20260807 15:06:59.486994 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 25310
I20260807 15:06:59.493058 25721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.492886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36269 (local address 127.0.0.1:39480)
0807 15:06:59.492955 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.493013 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:36269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:06:59.494277 25001 catalog_manager.cc:4584] TS ede1fb2929084d82bed7f84307fd4644 (127.18.129.1:36269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e637d4e8a6a849b5857a27a541f42f81: Network error: Client connection negotiation failed: client connection to 127.18.129.1:36269: connect: Connection refused (error 111)
I20260807 15:06:59.503871 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 25506
I20260807 15:06:59.514845 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 24978
2026-08-07T15:06:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2185 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T15:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:06:59Z Disabled control of system clock
I20260807 15:06:59.545475 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43911
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:43389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:43911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:06:59.625552 25774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:59.625748 25774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:59.625767 25774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:59.626828 25774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:06:59.626884 25774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:59.626896 25774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:59.626907 25774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:06:59.626919 25774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:06:59.628196 25774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:43911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:59.628381 25774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:59.628707 25774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:59.633592 25787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.633700 25783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.633790 25785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:59.633769 25774 server_base.cc:1034] running on GCE node
I20260807 15:06:59.634109 25774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:59.634351 25774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:59.635480 25774 hybrid_clock.cc:648] HybridClock initialized: now 1786115219635462 us; error 33 us; skew 500 ppm
I20260807 15:06:59.636732 25774 webserver.cc:460] Webserver started at http://127.18.129.62:45255/ using document root <none> and password file <none>
I20260807 15:06:59.637004 25774 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:59.637065 25774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:59.637187 25774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:59.638253 25774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "57175f57227243bcb043071500248bf6"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "fe2943154c27d1dc522821541286242d"
server_key_iv: "d7d8a94f57705626841f0dedfe16e737"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.638587 25774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "57175f57227243bcb043071500248bf6"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "fe2943154c27d1dc522821541286242d"
server_key_iv: "d7d8a94f57705626841f0dedfe16e737"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.639891 25774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:59.640543 25794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.640697 25774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:59.640784 25774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "57175f57227243bcb043071500248bf6"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "fe2943154c27d1dc522821541286242d"
server_key_iv: "d7d8a94f57705626841f0dedfe16e737"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.640842 25774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:59.644076 25774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:59.644311 25774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:59.644429 25774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:59.648234 25774 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:43911
I20260807 15:06:59.648264 25874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:43911 every 8 connection(s)
I20260807 15:06:59.648555 25774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:06:59.649169 25875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:06:59.649618 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 25774
I20260807 15:06:59.649704 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:06:59.649870 18948 external_mini_cluster.cc:1428] Setting key d403693f660dfbf678020b7e38ac0e07
I20260807 15:06:59.652509 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.650503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.18.129.62:43911)
0807 15:06:59.650716 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.650724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.650754 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:06:59.651323 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.651332 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.651385 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.651527 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:59.652098 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:59.652101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:59.652102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:59.652106 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:59.652152 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:59.652166 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:59.652244 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.652276 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.652330 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":67,"threads_started":1}
I20260807 15:06:59.652990 25875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6: Bootstrap starting.
I20260807 15:06:59.653611 25875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6: Neither blocks nor log segments found. Creating new log.
I20260807 15:06:59.653878 25875 log.cc:824] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:06:59.654476 25875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6: No bootstrap required, opened a new log
I20260807 15:06:59.655813 25875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57175f57227243bcb043071500248bf6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43911 } }
I20260807 15:06:59.655944 25875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:06:59.655977 25875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57175f57227243bcb043071500248bf6, State: Initialized, Role: FOLLOWER
I20260807 15:06:59.656085 25875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57175f57227243bcb043071500248bf6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43911 } }
I20260807 15:06:59.656157 25875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:06:59.656195 25875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:06:59.656242 25875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:06:59.656741 25875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57175f57227243bcb043071500248bf6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43911 } }
I20260807 15:06:59.656836 25875 leader_election.cc:302] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57175f57227243bcb043071500248bf6; no voters:
I20260807 15:06:59.656978 25875 leader_election.cc:288] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:06:59.657218 25881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:06:59.657224 25875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:06:59.657356 25881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [term 1 LEADER]: Becoming Leader. State: Replica: 57175f57227243bcb043071500248bf6, State: Running, Role: LEADER
I20260807 15:06:59.657485 25881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57175f57227243bcb043071500248bf6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43911 } }
I20260807 15:06:59.657805 25882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57175f57227243bcb043071500248bf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57175f57227243bcb043071500248bf6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43911 } } }
I20260807 15:06:59.657814 25883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57175f57227243bcb043071500248bf6. Latest consensus state: current_term: 1 leader_uuid: "57175f57227243bcb043071500248bf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57175f57227243bcb043071500248bf6" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43911 } } }
I20260807 15:06:59.657925 25882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:59.657933 25883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:06:59.658149 25894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:06:59.658627 25894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:06:59.660040 25894 catalog_manager.cc:1347] Generated new cluster ID: f1fb6df872084d35b0530a4fa6e46f48
I20260807 15:06:59.660084 25894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:06:59.666018 25894 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:06:59.666157 25894 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:06:59.669924 25894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6: Generated new TSK 0
I20260807 15:06:59.670091 25894 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:06:59.671651 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43911
--builtin_ntp_servers=127.18.129.20:43389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:06:59.752673 25914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:59.752879 25914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:59.752910 25914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:59.754055 25914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:59.754093 25914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:59.754127 25914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:06:59.755472 25914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:59.755697 25914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:59.755961 25914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:59.761229 25926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.761240 25922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:59.761368 25914 server_base.cc:1034] running on GCE node
W20260807 15:06:59.761269 25920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:59.761858 25914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:59.762080 25914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:59.763217 25914 hybrid_clock.cc:648] HybridClock initialized: now 1786115219763205 us; error 30 us; skew 500 ppm
I20260807 15:06:59.764477 25914 webserver.cc:460] Webserver started at http://127.18.129.1:37757/ using document root <none> and password file <none>
I20260807 15:06:59.764703 25914 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:59.764753 25914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:59.764878 25914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:59.765923 25914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "bf4604298e1a47abb27dc2373ae8f991"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "74534478b5458067855e762ac986b600"
server_key_iv: "e55972b937418f92f91c640cefc2b456"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.766261 25914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "bf4604298e1a47abb27dc2373ae8f991"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "74534478b5458067855e762ac986b600"
server_key_iv: "e55972b937418f92f91c640cefc2b456"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.767571 25914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:59.768276 25934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.768421 25914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:06:59.768510 25914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "bf4604298e1a47abb27dc2373ae8f991"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "74534478b5458067855e762ac986b600"
server_key_iv: "e55972b937418f92f91c640cefc2b456"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.768599 25914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:59.771876 25914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:59.772128 25914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:59.772239 25914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:59.772432 25914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:59.772744 25914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:59.772776 25914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.772796 25914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:59.772819 25914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.779299 25914 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:37071
I20260807 15:06:59.779331 26100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:37071 every 8 connection(s)
I20260807 15:06:59.779657 25914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:06:59.782326 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.780336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:51461 (local address 127.18.129.62:43911)
0807 15:06:59.780419 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.780422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.780617 (+ 195us) server_negotiation.cc:407] Connection header received
0807 15:06:59.781050 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.781051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.781056 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.781103 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:59.782147 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:59.782148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:59.782148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:59.782151 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:59.782165 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:59.782169 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:59.782211 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.782262 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.782273 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:06:59.782486 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.780314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43911 (local address 127.18.129.1:51461)
0807 15:06:59.780488 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.780522 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.780976 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.781164 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.781167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.781175 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:59.782011 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:06:59.782011 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:59.782040 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:06:59.782068 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:59.782211 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:59.782213 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:59.782215 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.782263 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.782308 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260807 15:06:59.782991 26103 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43911
I20260807 15:06:59.783082 26103 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:59.783198 26103 heartbeater.cc:511] Master 127.18.129.62:43911 requested a full tablet report, sending...
I20260807 15:06:59.783602 25825 ts_manager.cc:194] Registered new tserver with Master: bf4604298e1a47abb27dc2373ae8f991 (127.18.129.1:37071)
I20260807 15:06:59.787838 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 25914
I20260807 15:06:59.787920 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:06:59.788102 18948 external_mini_cluster.cc:1428] Setting key 5e796e529f6faa4daf745c00e3ac9c2a
I20260807 15:06:59.789021 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43911
--builtin_ntp_servers=127.18.129.20:43389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:06:59.871176 26106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:59.871371 26106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:59.871402 26106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:59.872586 26106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:59.872627 26106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:59.872651 26106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:06:59.873955 26106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:59.874171 26106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:59.874440 26106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:59.879413 26112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.879455 26117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.879542 26113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:59.879792 26106 server_base.cc:1034] running on GCE node
I20260807 15:06:59.880017 26106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:59.880262 26106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:59.881417 26106 hybrid_clock.cc:648] HybridClock initialized: now 1786115219881396 us; error 37 us; skew 500 ppm
I20260807 15:06:59.882910 26106 webserver.cc:460] Webserver started at http://127.18.129.2:41499/ using document root <none> and password file <none>
I20260807 15:06:59.883133 26106 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:59.883185 26106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:59.883322 26106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:06:59.884819 26106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "820ac2da246c4dddb351cd13f2257799"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "a9658491b54aa354edabdacbf455162d"
server_key_iv: "d2c08f118a9d208a217e76ee48754625"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.885243 26106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "820ac2da246c4dddb351cd13f2257799"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "a9658491b54aa354edabdacbf455162d"
server_key_iv: "d2c08f118a9d208a217e76ee48754625"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.886524 26106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:06:59.887209 26126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.887359 26106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.887447 26106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "820ac2da246c4dddb351cd13f2257799"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "a9658491b54aa354edabdacbf455162d"
server_key_iv: "d2c08f118a9d208a217e76ee48754625"
server_key_version: "encryptionkey@0"
I20260807 15:06:59.887512 26106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:06:59.890731 26106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:06:59.890986 26106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:06:59.891112 26106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:06:59.891304 26106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:06:59.891592 26106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:06:59.891628 26106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.891654 26106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:06:59.891677 26106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:06:59.897163 26106 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43635
I20260807 15:06:59.897221 26294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43635 every 8 connection(s)
I20260807 15:06:59.897507 26106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:06:59.900322 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.898170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:34653 (local address 127.18.129.62:43911)
0807 15:06:59.898215 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:06:59.898217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:06:59.898436 (+ 219us) server_negotiation.cc:407] Connection header received
0807 15:06:59.898921 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:06:59.898922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:06:59.898929 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:06:59.898988 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:06:59.900088 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:06:59.900090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:06:59.900091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:06:59.900095 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:06:59.900112 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:06:59.900117 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:06:59.900164 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:06:59.900235 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:06:59.900248 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:06:59.900504 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:06:59.898145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43911 (local address 127.18.129.2:34653)
0807 15:06:59.898327 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:06:59.898358 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 15:06:59.898821 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:06:59.899056 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:06:59.899061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:06:59.899070 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:06:59.899946 (+ 876us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:06:59.899946 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:06:59.899973 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:06:59.900000 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:06:59.900169 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:06:59.900175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:06:59.900178 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:06:59.900246 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:06:59.900305 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260807 15:06:59.900956 26297 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43911
I20260807 15:06:59.901034 26297 heartbeater.cc:464] Registering TS with master...
I20260807 15:06:59.901165 26297 heartbeater.cc:511] Master 127.18.129.62:43911 requested a full tablet report, sending...
I20260807 15:06:59.901548 25825 ts_manager.cc:194] Registered new tserver with Master: 820ac2da246c4dddb351cd13f2257799 (127.18.129.2:43635)
I20260807 15:06:59.905700 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 26106
I20260807 15:06:59.905776 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:06:59.905938 18948 external_mini_cluster.cc:1428] Setting key 834faebb9f60897ec781f0e1de7f3c07
I20260807 15:06:59.906888 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43911
--builtin_ntp_servers=127.18.129.20:43389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:06:59.987910 26301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:06:59.988109 26301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:06:59.988131 26301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:06:59.989264 26301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:06:59.989302 26301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:06:59.989326 26301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:06:59.990729 26301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:06:59.990975 26301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:06:59.991271 26301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:06:59.996353 26308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.996325 26307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:06:59.996322 26310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:06:59.996695 26301 server_base.cc:1034] running on GCE node
I20260807 15:06:59.996902 26301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:06:59.997145 26301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:06:59.998287 26301 hybrid_clock.cc:648] HybridClock initialized: now 1786115219998275 us; error 28 us; skew 500 ppm
I20260807 15:06:59.999563 26301 webserver.cc:460] Webserver started at http://127.18.129.3:40193/ using document root <none> and password file <none>
I20260807 15:06:59.999794 26301 fs_manager.cc:359] Metadata directory not provided
I20260807 15:06:59.999843 26301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:06:59.999997 26301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:00.001084 26301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "af43a8dacdae47efa7bb59a212c89b76"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "2b7591141f5f49aef68e45aff19756e0"
server_key_iv: "fab270ddcd5de632d8eac319b5b8cd32"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.001427 26301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "af43a8dacdae47efa7bb59a212c89b76"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "2b7591141f5f49aef68e45aff19756e0"
server_key_iv: "fab270ddcd5de632d8eac319b5b8cd32"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.002682 26301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:00.003413 26319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:00.003574 26301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:00.003664 26301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "af43a8dacdae47efa7bb59a212c89b76"
format_stamp: "Formatted at 2026-08-07 15:06:59 on dist-test-slave-5bn2"
server_key: "2b7591141f5f49aef68e45aff19756e0"
server_key_iv: "fab270ddcd5de632d8eac319b5b8cd32"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.003731 26301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:00.007081 26301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:00.007321 26301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:00.007438 26301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:00.007618 26301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:00.007894 26301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:00.007934 26301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:00.007962 26301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:00.007987 26301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:00.013366 26301 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:39229
I20260807 15:07:00.013423 26490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:39229 every 8 connection(s)
I20260807 15:07:00.013692 26301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:07:00.016372 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.014342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:40545 (local address 127.18.129.62:43911)
0807 15:07:00.014389 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.014391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.014613 (+ 222us) server_negotiation.cc:407] Connection header received
0807 15:07:00.015107 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.015108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.015113 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.015170 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.016182 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.016183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.016184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.016187 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.016203 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.016208 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.016250 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.016307 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.016317 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:07:00.016619 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.014302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43911 (local address 127.18.129.3:40545)
0807 15:07:00.014497 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.014536 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.015025 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.015220 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.015223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.015231 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:00.016056 (+ 825us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:00.016056 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:00.016083 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:00.016111 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:00.016255 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:00.016258 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:00.016260 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:07:00.016309 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:07:00.016355 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260807 15:07:00.017102 26491 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43911
I20260807 15:07:00.017184 26491 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:00.017318 26491 heartbeater.cc:511] Master 127.18.129.62:43911 requested a full tablet report, sending...
I20260807 15:07:00.017645 25825 ts_manager.cc:194] Registered new tserver with Master: af43a8dacdae47efa7bb59a212c89b76 (127.18.129.3:39229)
I20260807 15:07:00.022151 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 26301
I20260807 15:07:00.022246 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:07:00.022397 18948 external_mini_cluster.cc:1428] Setting key 015fbb3e35756384dca46f85dbbd7cca
I20260807 15:07:00.023186 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:07:00.025403 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.024403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.18.129.62:43911)
0807 15:07:00.024447 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.024449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.024587 (+ 138us) server_negotiation.cc:407] Connection header received
0807 15:07:00.024649 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.024650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.024656 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.024697 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.025199 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.025201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.025202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.025205 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.025219 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.025224 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.025270 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.025319 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.025332 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:07:00.026729 25825 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:07:00.028316 25825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:00.031963 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.030215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:39229 (local address 127.0.0.1:49144)
0807 15:07:00.030394 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.030435 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.030492 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.030875 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.030881 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.030887 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:00.031542 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:00.031543 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:00.031558 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:00.031580 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:00.031787 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:00.031793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:00.031796 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:00.031855 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:07:00.031868 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":81,"threads_started":1}
I20260807 15:07:00.031980 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.030559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.18.129.3:39229)
0807 15:07:00.030724 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.030728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.030744 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:07:00.030763 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.030764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.030772 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.030835 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.031641 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.031649 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.031650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.031654 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.031688 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.031710 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.031763 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.031875 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.031893 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260807 15:07:00.032109 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.030367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.18.129.1:37071)
0807 15:07:00.031234 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.031238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.031248 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:07:00.031273 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.031275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.031282 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.031350 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.031746 (+ 396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.031754 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.031754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.031758 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.031789 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.031813 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.031865 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.032002 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.032017 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":94,"threads_started":1}
I20260807 15:07:00.032189 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.030554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.18.129.2:43635)
0807 15:07:00.030743 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.030747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.030754 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:07:00.030776 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.030777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.030784 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.030866 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.031674 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.031683 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.031683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.031687 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.031723 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.031747 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.031803 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.032094 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.032110 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260807 15:07:00.032045 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.030216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:37071 (local address 127.0.0.1:41512)
0807 15:07:00.030573 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.030583 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.030629 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.031362 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.031363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.031366 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:00.031673 (+ 307us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:00.031673 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:00.031679 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:00.031687 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:00.031893 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:00.031894 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:00.031895 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:07:00.031937 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:07:00.031949 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":83,"threads_started":1}
I20260807 15:07:00.031963 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.030435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43635 (local address 127.0.0.1:33982)
0807 15:07:00.030590 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.030599 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.030632 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.030880 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.030887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.030890 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:00.031550 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:00.031551 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:00.031558 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:00.031588 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:00.031805 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:00.031806 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:00.031806 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 15:07:00.031853 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:07:00.031870 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260807 15:07:00.033082 26392 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6208cb665046cca42439d86849b6dd (DEFAULT_TABLE table=test-table [id=38836cc39787443ea422d0659a7fcb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:00.033083 26192 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6208cb665046cca42439d86849b6dd (DEFAULT_TABLE table=test-table [id=38836cc39787443ea422d0659a7fcb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:00.033097 26008 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6208cb665046cca42439d86849b6dd (DEFAULT_TABLE table=test-table [id=38836cc39787443ea422d0659a7fcb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:00.033461 26008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6208cb665046cca42439d86849b6dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:00.033463 26192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6208cb665046cca42439d86849b6dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:00.033463 26392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6208cb665046cca42439d86849b6dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:00.036075 26508 tablet_bootstrap.cc:492] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991: Bootstrap starting.
I20260807 15:07:00.036659 26510 tablet_bootstrap.cc:492] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: Bootstrap starting.
I20260807 15:07:00.036796 26508 tablet_bootstrap.cc:654] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:00.037132 26508 log.cc:824] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:00.037309 26510 tablet_bootstrap.cc:654] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:00.037591 26510 log.cc:824] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:00.037919 26509 tablet_bootstrap.cc:492] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: Bootstrap starting.
I20260807 15:07:00.037993 26508 tablet_bootstrap.cc:492] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991: No bootstrap required, opened a new log
I20260807 15:07:00.038147 26508 ts_tablet_manager.cc:1397] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:07:00.038354 26510 tablet_bootstrap.cc:492] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: No bootstrap required, opened a new log
I20260807 15:07:00.038419 26510 ts_tablet_manager.cc:1397] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:00.038565 26509 tablet_bootstrap.cc:654] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:00.038913 26509 log.cc:824] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:00.039690 26509 tablet_bootstrap.cc:492] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: No bootstrap required, opened a new log
I20260807 15:07:00.039759 26509 ts_tablet_manager.cc:1397] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:07:00.040514 26510 raft_consensus.cc:348] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.040653 26510 raft_consensus.cc:374] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:00.040684 26510 raft_consensus.cc:729] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820ac2da246c4dddb351cd13f2257799, State: Initialized, Role: FOLLOWER
I20260807 15:07:00.040671 26508 raft_consensus.cc:348] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.040833 26508 raft_consensus.cc:374] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:00.040818 26510 consensus_queue.cc:260] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.040875 26508 raft_consensus.cc:729] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf4604298e1a47abb27dc2373ae8f991, State: Initialized, Role: FOLLOWER
I20260807 15:07:00.041024 26508 consensus_queue.cc:260] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.041090 26297 heartbeater.cc:503] Master 127.18.129.62:43911 was elected leader, sending a full tablet report...
I20260807 15:07:00.041090 26510 ts_tablet_manager.cc:1428] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:07:00.041275 26508 ts_tablet_manager.cc:1428] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:07:00.041723 26103 heartbeater.cc:503] Master 127.18.129.62:43911 was elected leader, sending a full tablet report...
I20260807 15:07:00.042188 26509 raft_consensus.cc:348] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.042342 26509 raft_consensus.cc:374] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:00.042390 26509 raft_consensus.cc:729] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af43a8dacdae47efa7bb59a212c89b76, State: Initialized, Role: FOLLOWER
I20260807 15:07:00.042510 26509 consensus_queue.cc:260] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.042755 26509 ts_tablet_manager.cc:1428] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:07:00.042805 26491 heartbeater.cc:503] Master 127.18.129.62:43911 was elected leader, sending a full tablet report...
I20260807 15:07:00.095503 26516 raft_consensus.cc:482] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:00.095657 26516 raft_consensus.cc:504] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.096019 26516 leader_election.cc:288] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af43a8dacdae47efa7bb59a212c89b76 (127.18.129.3:39229), 820ac2da246c4dddb351cd13f2257799 (127.18.129.2:43635)
I20260807 15:07:00.097223 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.096203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43635 (local address 127.18.129.1:42175)
0807 15:07:00.096287 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.096301 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.096353 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.096508 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.096509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.096512 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:00.096917 (+ 405us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:00.096918 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:00.096925 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:00.096937 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:00.097110 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:00.097111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:00.097111 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 15:07:00.097156 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:07:00.097169 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:07:00.097251 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.096321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:42175 (local address 127.18.129.2:43635)
0807 15:07:00.096366 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.096369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.096378 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:07:00.096408 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.096409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.096415 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.096483 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.097015 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.097017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.097017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.097022 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.097041 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.097045 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.097095 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.097162 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.097176 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:07:00.097506 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.096203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:39229 (local address 127.18.129.1:60637)
0807 15:07:00.096447 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.096453 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.096495 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.096616 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.096616 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.096619 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:00.097073 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:00.097073 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:00.097079 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:00.097090 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:00.097259 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:00.097259 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:00.097260 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:07:00.097420 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 15:07:00.097432 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":80,"thread_start_us":111,"threads_started":1}
I20260807 15:07:00.097553 26219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6208cb665046cca42439d86849b6dd" candidate_uuid: "bf4604298e1a47abb27dc2373ae8f991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "820ac2da246c4dddb351cd13f2257799" is_pre_election: true
I20260807 15:07:00.097718 26219 raft_consensus.cc:2454] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf4604298e1a47abb27dc2373ae8f991 in term 0.
I20260807 15:07:00.097937 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.096289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:60637 (local address 127.18.129.3:39229)
0807 15:07:00.096348 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.096350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.096519 (+ 169us) server_negotiation.cc:407] Connection header received
0807 15:07:00.096551 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.096552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.096558 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.096599 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.097173 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.097174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.097175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.097180 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.097197 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.097202 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.097254 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.097838 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.097854 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:07:00.097942 25939 leader_election.cc:302] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 820ac2da246c4dddb351cd13f2257799, bf4604298e1a47abb27dc2373ae8f991; no voters:
I20260807 15:07:00.098078 26516 raft_consensus.cc:2781] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:00.098132 26516 raft_consensus.cc:482] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:00.098152 26516 raft_consensus.cc:3037] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:00.098223 26422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6208cb665046cca42439d86849b6dd" candidate_uuid: "bf4604298e1a47abb27dc2373ae8f991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af43a8dacdae47efa7bb59a212c89b76" is_pre_election: true
I20260807 15:07:00.098357 26422 raft_consensus.cc:2454] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf4604298e1a47abb27dc2373ae8f991 in term 0.
I20260807 15:07:00.099009 26516 raft_consensus.cc:504] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.099136 26516 leader_election.cc:288] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [CANDIDATE]: Term 1 election: Requested vote from peers af43a8dacdae47efa7bb59a212c89b76 (127.18.129.3:39229), 820ac2da246c4dddb351cd13f2257799 (127.18.129.2:43635)
I20260807 15:07:00.099282 26219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6208cb665046cca42439d86849b6dd" candidate_uuid: "bf4604298e1a47abb27dc2373ae8f991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "820ac2da246c4dddb351cd13f2257799"
I20260807 15:07:00.099344 26422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6208cb665046cca42439d86849b6dd" candidate_uuid: "bf4604298e1a47abb27dc2373ae8f991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af43a8dacdae47efa7bb59a212c89b76"
I20260807 15:07:00.099370 26219 raft_consensus.cc:3037] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:00.099470 26422 raft_consensus.cc:3037] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:00.100267 26219 raft_consensus.cc:2454] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf4604298e1a47abb27dc2373ae8f991 in term 1.
I20260807 15:07:00.100291 26422 raft_consensus.cc:2454] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf4604298e1a47abb27dc2373ae8f991 in term 1.
I20260807 15:07:00.100418 25939 leader_election.cc:302] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 820ac2da246c4dddb351cd13f2257799, bf4604298e1a47abb27dc2373ae8f991; no voters:
I20260807 15:07:00.100536 26516 raft_consensus.cc:2781] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:00.100673 26516 raft_consensus.cc:686] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [term 1 LEADER]: Becoming Leader. State: Replica: bf4604298e1a47abb27dc2373ae8f991, State: Running, Role: LEADER
I20260807 15:07:00.100750 26516 consensus_queue.cc:237] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } }
I20260807 15:07:00.101609 25823 catalog_manager.cc:5495] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf4604298e1a47abb27dc2373ae8f991 (127.18.129.1). New cstate: current_term: 1 leader_uuid: "bf4604298e1a47abb27dc2373ae8f991" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf4604298e1a47abb27dc2373ae8f991" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 } health_report { overall_health: UNKNOWN } } }
I20260807 15:07:00.123754 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.122781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.18.129.1:37071)
0807 15:07:00.122865 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.122867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.122877 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:07:00.122918 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.122918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.122932 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.122978 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.123536 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.123537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.123538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.123542 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.123561 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.123565 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.123613 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.123659 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.123672 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:07:00.125486 26219 raft_consensus.cc:1261] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 1 FOLLOWER]: Refusing update from remote peer bf4604298e1a47abb27dc2373ae8f991: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:00.125552 26422 raft_consensus.cc:1261] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 1 FOLLOWER]: Refusing update from remote peer bf4604298e1a47abb27dc2373ae8f991: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:00.125903 26516 consensus_queue.cc:1035] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af43a8dacdae47efa7bb59a212c89b76" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 39229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:00.126061 26520 consensus_queue.cc:1035] T 0d6208cb665046cca42439d86849b6dd P bf4604298e1a47abb27dc2373ae8f991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "820ac2da246c4dddb351cd13f2257799" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:00.131331 25823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:00.131436 25823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:00.132182 25823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57175f57227243bcb043071500248bf6: Sending DeleteTablet for 3 replicas of tablet 0d6208cb665046cca42439d86849b6dd
I20260807 15:07:00.132520 26392 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6208cb665046cca42439d86849b6dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:00 UTC) from {username='slave'} at 127.0.0.1:49144
I20260807 15:07:00.132560 26008 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6208cb665046cca42439d86849b6dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:00 UTC) from {username='slave'} at 127.0.0.1:41512
I20260807 15:07:00.132619 26192 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6208cb665046cca42439d86849b6dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:00 UTC) from {username='slave'} at 127.0.0.1:33982
I20260807 15:07:00.132825 26541 tablet_replica.cc:330] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: stopping tablet replica
I20260807 15:07:00.132936 26541 raft_consensus.cc:2229] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:00.132994 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 25914
I20260807 15:07:00.133036 26541 raft_consensus.cc:2258] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:00.137143 26541 ts_tablet_manager.cc:1905] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:00.139091 26541 ts_tablet_manager.cc:1918] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:00.139173 26541 log.cc:1196] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/0d6208cb665046cca42439d86849b6dd
I20260807 15:07:00.139420 26541 ts_tablet_manager.cc:1939] T 0d6208cb665046cca42439d86849b6dd P 820ac2da246c4dddb351cd13f2257799: Deleting consensus metadata
I20260807 15:07:00.139815 25799 catalog_manager.cc:4841] TS 820ac2da246c4dddb351cd13f2257799 (127.18.129.2:43635): tablet 0d6208cb665046cca42439d86849b6dd (table test-table [id=38836cc39787443ea422d0659a7fcb32]) successfully deleted
I20260807 15:07:00.142465 26539 tablet_replica.cc:330] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: stopping tablet replica
I20260807 15:07:00.142567 26539 raft_consensus.cc:2229] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:00.142652 26539 raft_consensus.cc:2258] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:00.143078 26539 ts_tablet_manager.cc:1905] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:00.144608 26539 ts_tablet_manager.cc:1918] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:00.144682 26539 log.cc:1196] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/0d6208cb665046cca42439d86849b6dd
I20260807 15:07:00.144963 26539 ts_tablet_manager.cc:1939] T 0d6208cb665046cca42439d86849b6dd P af43a8dacdae47efa7bb59a212c89b76: Deleting consensus metadata
I20260807 15:07:00.145351 25797 catalog_manager.cc:4841] TS af43a8dacdae47efa7bb59a212c89b76 (127.18.129.3:39229): tablet 0d6208cb665046cca42439d86849b6dd (table test-table [id=38836cc39787443ea422d0659a7fcb32]) successfully deleted
W20260807 15:07:00.147068 25799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:37071 (error 108)
I20260807 15:07:00.147286 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 26106
I20260807 15:07:00.150714 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.150572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:37071 (local address 127.0.0.1:41530)
0807 15:07:00.150629 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.150679 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:37071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 15:07:00.150825 25799 catalog_manager.cc:4584] TS bf4604298e1a47abb27dc2373ae8f991 (127.18.129.1:37071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0d6208cb665046cca42439d86849b6dd: Network error: Client connection negotiation failed: client connection to 127.18.129.1:37071: connect: Connection refused (error 111)
I20260807 15:07:00.160755 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 26301
I20260807 15:07:00.170145 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 25774
2026-08-07T15:07:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (656 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 15:07:00.194501 18948 mini_oidc.cc:152] Starting JWKS server
I20260807 15:07:00.195854 18948 webserver.cc:460] Webserver started at https://127.0.0.1:38309/ using document root <none> and password file <none>
I20260807 15:07:00.195958 18948 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:07:00.196135 18948 webserver.cc:460] Webserver started at http://127.0.0.1:45355/ using document root <none> and password file <none>
2026-08-07T15:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:00Z Disabled control of system clock
I20260807 15:07:00.204677 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43637
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:41217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:43637
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38309/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/testchainca.cert with env {}
W20260807 15:07:00.286053 26552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:00.286257 26552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:00.286278 26552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:00.287503 26552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:00.287544 26552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:00.287555 26552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:00.287567 26552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:00.287577 26552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:07:00.287587 26552 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:07:00.287597 26552 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38309/jwks.json
I20260807 15:07:00.289152 26552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:43637
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38309/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:00.289350 26552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:00.289520 26552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:00.294422 26557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:00.294624 26552 server_base.cc:1034] running on GCE node
W20260807 15:07:00.294438 26560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:00.294562 26558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:00.295037 26552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:00.295315 26552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:00.296455 26552 hybrid_clock.cc:648] HybridClock initialized: now 1786115220296442 us; error 30 us; skew 500 ppm
I20260807 15:07:00.297672 26552 webserver.cc:460] Webserver started at http://127.18.129.62:35537/ using document root <none> and password file <none>
I20260807 15:07:00.297906 26552 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:00.297955 26552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:00.298118 26552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:00.299198 26552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "2860940fbc474edfa8e53b01065d0e54"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "88eceb140033d1e9c42b8dec74b83914"
server_key_iv: "d54e4e813d89c545faeb3deff070ef97"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.299527 26552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "2860940fbc474edfa8e53b01065d0e54"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "88eceb140033d1e9c42b8dec74b83914"
server_key_iv: "d54e4e813d89c545faeb3deff070ef97"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.300731 26552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:07:00.301499 26568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:00.301659 26552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:00.301772 26552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "2860940fbc474edfa8e53b01065d0e54"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "88eceb140033d1e9c42b8dec74b83914"
server_key_iv: "d54e4e813d89c545faeb3deff070ef97"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.301852 26552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:00.325373 26552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:00.325749 26552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:00.325887 26552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:00.330214 26552 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:43637
I20260807 15:07:00.330262 26646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:43637 every 8 connection(s)
I20260807 15:07:00.330621 26552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:00.331796 26647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:00.334177 26647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54: Bootstrap starting.
I20260807 15:07:00.334758 26647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:00.335090 26647 log.cc:824] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:00.335798 26647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54: No bootstrap required, opened a new log
I20260807 15:07:00.337826 26647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2860940fbc474edfa8e53b01065d0e54" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43637 } }
I20260807 15:07:00.338025 26647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:00.338068 26647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2860940fbc474edfa8e53b01065d0e54, State: Initialized, Role: FOLLOWER
I20260807 15:07:00.338222 26647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2860940fbc474edfa8e53b01065d0e54" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43637 } }
I20260807 15:07:00.338304 26647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:00.338335 26647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:00.338375 26647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:00.339061 26647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2860940fbc474edfa8e53b01065d0e54" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43637 } }
I20260807 15:07:00.339174 26647 leader_election.cc:302] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2860940fbc474edfa8e53b01065d0e54; no voters:
I20260807 15:07:00.339238 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 26552
I20260807 15:07:00.339342 26647 leader_election.cc:288] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:00.339457 26659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:00.339613 26647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:00.339624 26659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [term 1 LEADER]: Becoming Leader. State: Replica: 2860940fbc474edfa8e53b01065d0e54, State: Running, Role: LEADER
I20260807 15:07:00.339751 26659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2860940fbc474edfa8e53b01065d0e54" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43637 } }
I20260807 15:07:00.339922 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:00.340075 26659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2860940fbc474edfa8e53b01065d0e54. Latest consensus state: current_term: 1 leader_uuid: "2860940fbc474edfa8e53b01065d0e54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2860940fbc474edfa8e53b01065d0e54" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43637 } } }
I20260807 15:07:00.340148 26659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:00.340166 18948 external_mini_cluster.cc:1428] Setting key a2c6c13e2a19fbc3ee01a7c65e92133e
I20260807 15:07:00.340301 26660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2860940fbc474edfa8e53b01065d0e54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2860940fbc474edfa8e53b01065d0e54" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43637 } } }
I20260807 15:07:00.340463 26660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:00.340356 26669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:00.341336 26669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:00.345913 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.340758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.18.129.62:43637)
0807 15:07:00.341098 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.341108 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.341142 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:07:00.341264 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.341269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.341323 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.341462 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.341693 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.343090 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.343865 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.344486 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.344618 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.345183 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.345188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.345188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.345392 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.345439 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.345467 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.345536 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.345616 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.345700 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":72,"threads_started":1}
I20260807 15:07:00.346951 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:07:00.347286 26669 catalog_manager.cc:1347] Generated new cluster ID: 09d52a52da27463fa8083b282b0400ee
I20260807 15:07:00.347344 26669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:00.350924 26669 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:00.351426 26669 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:00.361738 26669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2860940fbc474edfa8e53b01065d0e54: Generated new TSK 0
I20260807 15:07:00.361940 26669 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:07:00.390547 26696 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 15:07:00.390762 26696 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:07:00.390951 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.368210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.18.129.62:43637)
0807 15:07:00.368269 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.368271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.368424 (+ 153us) server_negotiation.cc:407] Connection header received
0807 15:07:00.368557 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.368558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.368571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.368626 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:07:00.368757 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.369249 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.370010 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.370218 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.370272 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.370474 (+ 202us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:07:00.390530 (+ 20056us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:07:00.390776 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39402: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 15:07:00.391021 26679 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39402: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 15:07:00.391271 26694 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:00.368130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43637 (local address 127.0.0.1:39402)
0807 15:07:00.368306 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.368317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.368460 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.368630 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.368632 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.368633 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:07:00.368700 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:00.368702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.369322 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.369323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:00.369931 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:00.369934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.370289 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.370290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:00.370350 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.370378 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:07:00.390692 (+ 20314us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:07:00.390821 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.129.62:43637: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.129.62:43637: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 15:07:00.391647 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 26552
2026-08-07T15:07:00Z chronyd exiting
I20260807 15:07:00.431630 18948 test_util.cc:175] -----------------------------------------------
I20260807 15:07:00.431715 18948 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115197351496-18948-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 15:07:00.432515 18948 mini_oidc.cc:152] Starting JWKS server
I20260807 15:07:00.433192 18948 webserver.cc:460] Webserver started at https://127.0.0.1:46035/ using document root <none> and password file <none>
I20260807 15:07:00.433235 18948 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:07:00.433369 18948 webserver.cc:460] Webserver started at http://127.0.0.1:44087/ using document root <none> and password file <none>
2026-08-07T15:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:00Z Disabled control of system clock
I20260807 15:07:00.441738 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45053
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:45053
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46035/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/testchainca.cert with env {}
W20260807 15:07:00.521685 26704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:00.521896 26704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:00.521924 26704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:00.523069 26704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:00.523111 26704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:00.523128 26704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:00.523144 26704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:00.523157 26704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:07:00.523173 26704 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:07:00.523192 26704 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46035/jwks.json
I20260807 15:07:00.524698 26704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:45053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46035/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:00.524946 26704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:00.525138 26704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:00.529803 26711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:00.529889 26712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:00.529899 26704 server_base.cc:1034] running on GCE node
W20260807 15:07:00.529825 26717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:00.530259 26704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:00.530529 26704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:00.531662 26704 hybrid_clock.cc:648] HybridClock initialized: now 1786115220531644 us; error 34 us; skew 500 ppm
I20260807 15:07:00.532840 26704 webserver.cc:460] Webserver started at http://127.18.129.62:39625/ using document root <none> and password file <none>
I20260807 15:07:00.533082 26704 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:00.533133 26704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:00.533252 26704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:00.534301 26704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "d9e60ab8b9d04f36ae542c3bb520b789"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "a0d2385e87fa8880aa4826969dacde9f"
server_key_iv: "102d152a65fc35dc2dd7576c5fe4c9d6"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.534623 26704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "d9e60ab8b9d04f36ae542c3bb520b789"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "a0d2385e87fa8880aa4826969dacde9f"
server_key_iv: "102d152a65fc35dc2dd7576c5fe4c9d6"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.535852 26704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:07:00.536494 26724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:00.536626 26704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:07:00.536701 26704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "d9e60ab8b9d04f36ae542c3bb520b789"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "a0d2385e87fa8880aa4826969dacde9f"
server_key_iv: "102d152a65fc35dc2dd7576c5fe4c9d6"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.536761 26704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:00.549546 26704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:00.549829 26704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:00.549955 26704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:00.553829 26704 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:45053
I20260807 15:07:00.553864 26805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:45053 every 8 connection(s)
I20260807 15:07:00.554188 26704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:00.554909 26807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:00.555940 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 26704
I20260807 15:07:00.556030 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:00.556191 18948 external_mini_cluster.cc:1428] Setting key 8af81274add0a2aa80620cbcb786f4b5
I20260807 15:07:00.558432 26807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789: Bootstrap starting.
I20260807 15:07:00.559162 26807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:00.559458 26807 log.cc:824] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:00.560153 26807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789: No bootstrap required, opened a new log
I20260807 15:07:00.562000 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.556676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.18.129.62:45053)
0807 15:07:00.556889 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.556899 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.556935 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:07:00.557453 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.557461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.557543 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.557697 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.557852 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.559182 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.559976 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.560561 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.560680 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.561288 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.561293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.561293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.561492 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.561536 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.561551 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.561619 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.561678 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.561761 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260807 15:07:00.562348 26807 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e60ab8b9d04f36ae542c3bb520b789" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45053 } }
I20260807 15:07:00.562516 26807 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:00.562570 26807 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9e60ab8b9d04f36ae542c3bb520b789, State: Initialized, Role: FOLLOWER
I20260807 15:07:00.562680 26807 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e60ab8b9d04f36ae542c3bb520b789" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45053 } }
I20260807 15:07:00.562762 26807 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:00.562788 26807 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:00.562836 26807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:00.563146 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:07:00.563526 26807 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e60ab8b9d04f36ae542c3bb520b789" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45053 } }
I20260807 15:07:00.563650 26807 leader_election.cc:302] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9e60ab8b9d04f36ae542c3bb520b789; no voters:
I20260807 15:07:00.563807 26807 leader_election.cc:288] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:00.563932 26814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:00.564118 26814 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [term 1 LEADER]: Becoming Leader. State: Replica: d9e60ab8b9d04f36ae542c3bb520b789, State: Running, Role: LEADER
I20260807 15:07:00.564128 26807 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:00.564312 26814 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e60ab8b9d04f36ae542c3bb520b789" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45053 } }
I20260807 15:07:00.564798 26817 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9e60ab8b9d04f36ae542c3bb520b789. Latest consensus state: current_term: 1 leader_uuid: "d9e60ab8b9d04f36ae542c3bb520b789" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e60ab8b9d04f36ae542c3bb520b789" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45053 } } }
I20260807 15:07:00.564808 26815 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9e60ab8b9d04f36ae542c3bb520b789" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e60ab8b9d04f36ae542c3bb520b789" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45053 } } }
I20260807 15:07:00.564908 26817 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:00.564913 26815 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:00.565231 26828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:00.565730 26828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:00.567152 26828 catalog_manager.cc:1347] Generated new cluster ID: 81784e62029b400e84560f9984b9686d
I20260807 15:07:00.567210 26828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:00.588421 26828 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:00.589154 26828 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:00.592393 26828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9e60ab8b9d04f36ae542c3bb520b789: Generated new TSK 0
I20260807 15:07:00.592594 26828 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:07:00.611752 26852 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 15:07:00.611827 26852 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:07:00.612221 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.597916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.18.129.62:45053)
0807 15:07:00.597964 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.597966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.598045 (+ 79us) server_negotiation.cc:407] Connection header received
0807 15:07:00.598114 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.598115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.598129 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.598222 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:07:00.598330 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.599158 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.599881 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.600191 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.600251 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.600378 (+ 127us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:07:00.611970 (+ 11592us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46035/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:07:00.612133 (+ 163us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39072: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46035/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 15:07:00.612268 26851 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:00.597843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45053 (local address 127.0.0.1:39072)
0807 15:07:00.597978 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.597988 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.598046 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.598218 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.598219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.598220 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:07:00.598274 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:00.598276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.599230 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.599230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:00.599809 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:00.599811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.600259 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.600260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:00.600303 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.600308 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:07:00.612086 (+ 11778us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46035/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:07:00.612192 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.129.62:45053: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46035/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
W20260807 15:07:00.612289 26811 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39072: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46035/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260807 15:07:00.612566 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 26704
2026-08-07T15:07:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 15:07:00.655383 18948 mini_oidc.cc:152] Starting JWKS server
I20260807 15:07:00.656057 18948 webserver.cc:460] Webserver started at https://127.0.0.1:36139/ using document root <none> and password file <none>
I20260807 15:07:00.656102 18948 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:07:00.656243 18948 webserver.cc:460] Webserver started at http://127.0.0.1:40169/ using document root <none> and password file <none>
2026-08-07T15:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:00Z Disabled control of system clock
I20260807 15:07:00.664736 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43909
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:37531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:43909
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36139/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:07:00.744135 26860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:00.744346 26860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:00.744375 26860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:00.745508 26860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:00.745548 26860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:00.745564 26860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:00.745590 26860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:00.745605 26860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:07:00.745625 26860 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:07:00.745641 26860 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36139/jwks.json
I20260807 15:07:00.747030 26860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:37531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:43909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36139/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:00.747274 26860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:00.747526 26860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:00.752408 26868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:00.752445 26871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:00.752452 26860 server_base.cc:1034] running on GCE node
W20260807 15:07:00.752419 26869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:00.752945 26860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:00.753188 26860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:00.754316 26860 hybrid_clock.cc:648] HybridClock initialized: now 1786115220754298 us; error 31 us; skew 500 ppm
I20260807 15:07:00.755510 26860 webserver.cc:460] Webserver started at http://127.18.129.62:39931/ using document root <none> and password file <none>
I20260807 15:07:00.755746 26860 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:00.755791 26860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:00.755909 26860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:00.756919 26860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "deb5eca832534dc0934e6ef20a01ca36"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "990b036042c348bec03d48c72494aaaf"
server_key_iv: "6cf321744d847276eeb5516fe4293c6f"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.757249 26860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "deb5eca832534dc0934e6ef20a01ca36"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "990b036042c348bec03d48c72494aaaf"
server_key_iv: "6cf321744d847276eeb5516fe4293c6f"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.758415 26860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:07:00.759150 26880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:00.759313 26860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:00.759392 26860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "deb5eca832534dc0934e6ef20a01ca36"
format_stamp: "Formatted at 2026-08-07 15:07:00 on dist-test-slave-5bn2"
server_key: "990b036042c348bec03d48c72494aaaf"
server_key_iv: "6cf321744d847276eeb5516fe4293c6f"
server_key_version: "encryptionkey@0"
I20260807 15:07:00.759444 26860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:00.789175 26860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:00.789498 26860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:00.789615 26860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:00.793529 26860 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:43909
I20260807 15:07:00.793612 26961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:43909 every 8 connection(s)
I20260807 15:07:00.793921 26860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:00.794616 26962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:00.797539 26962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36: Bootstrap starting.
I20260807 15:07:00.798139 26962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:00.798408 26962 log.cc:824] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:00.799031 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 26860
I20260807 15:07:00.799106 26962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36: No bootstrap required, opened a new log
I20260807 15:07:00.799127 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:00.799273 18948 external_mini_cluster.cc:1428] Setting key b321294a68e96294ea1762ed0ebe8085
I20260807 15:07:00.801890 26962 raft_consensus.cc:348] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb5eca832534dc0934e6ef20a01ca36" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43909 } }
I20260807 15:07:00.802070 26962 raft_consensus.cc:374] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:00.802100 26962 raft_consensus.cc:729] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deb5eca832534dc0934e6ef20a01ca36, State: Initialized, Role: FOLLOWER
I20260807 15:07:00.802222 26962 consensus_queue.cc:260] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb5eca832534dc0934e6ef20a01ca36" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43909 } }
I20260807 15:07:00.802304 26962 raft_consensus.cc:388] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:00.802345 26962 raft_consensus.cc:482] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:00.802393 26962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:00.802969 26962 raft_consensus.cc:504] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb5eca832534dc0934e6ef20a01ca36" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43909 } }
I20260807 15:07:00.803084 26962 leader_election.cc:302] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: deb5eca832534dc0934e6ef20a01ca36; no voters:
I20260807 15:07:00.803249 26962 leader_election.cc:288] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:00.803287 26968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:00.803427 26968 raft_consensus.cc:686] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [term 1 LEADER]: Becoming Leader. State: Replica: deb5eca832534dc0934e6ef20a01ca36, State: Running, Role: LEADER
I20260807 15:07:00.803486 26962 sys_catalog.cc:564] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:00.803516 26968 consensus_queue.cc:237] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb5eca832534dc0934e6ef20a01ca36" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43909 } }
I20260807 15:07:00.803846 26968 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader deb5eca832534dc0934e6ef20a01ca36. Latest consensus state: current_term: 1 leader_uuid: "deb5eca832534dc0934e6ef20a01ca36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb5eca832534dc0934e6ef20a01ca36" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43909 } } }
I20260807 15:07:00.803920 26968 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:00.804118 26971 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "deb5eca832534dc0934e6ef20a01ca36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb5eca832534dc0934e6ef20a01ca36" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43909 } } }
I20260807 15:07:00.804251 26971 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:00.804594 26983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:00.805270 26967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.799741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34572 (local address 127.18.129.62:43909)
0807 15:07:00.799941 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.799951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.799985 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:07:00.800452 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.800461 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.800548 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.800697 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:00.800869 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.802213 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.803044 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.803656 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.803808 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.804466 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:00.804470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:00.804470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:00.804681 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:00.804726 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:00.804742 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:00.804808 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:00.804999 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 15:07:00.805080 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260807 15:07:00.806010 26983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:00.806214 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:07:00.807621 26983 catalog_manager.cc:1347] Generated new cluster ID: b57a259234b641278f90976c4d7017b3
I20260807 15:07:00.807674 26983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:00.848328 26983 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:00.848901 26983 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:00.854698 26983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P deb5eca832534dc0934e6ef20a01ca36: Generated new TSK 0
I20260807 15:07:00.854907 26983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:07:00.866387 26967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:00.862715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.18.129.62:43909)
0807 15:07:00.862763 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:00.862765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:00.862892 (+ 127us) server_negotiation.cc:407] Connection header received
0807 15:07:00.862979 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:00.862980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:00.862996 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:00.863076 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:07:00.863183 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.863918 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.864767 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.865056 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.865132 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.865250 (+ 118us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:07:00.866181 (+ 931us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36139/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:07:00.866314 (+ 133us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34584: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36139/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 15:07:00.866456 26967 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34584: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36139/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 15:07:00.868289 27002 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:00.862636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43909 (local address 127.0.0.1:34584)
0807 15:07:00.862810 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:00.862820 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:00.862909 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:00.863074 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:00.863075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:00.863077 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:07:00.863128 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:00.863131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.863999 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.863999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:00.864689 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:00.864692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:00.865130 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:00.865130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:00.865165 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:00.865170 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:07:00.866287 (+ 1117us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36139/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:07:00.868187 (+ 1900us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.129.62:43909: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36139/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":1702,"thread_start_us":118,"threads_started":1}
I20260807 15:07:00.868649 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 26860
E20260807 15:07:00.878684 27006 webserver.cc:533] Webserver: sslize failed:
E20260807 15:07:00.878755 27006 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T15:07:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (255 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T15:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:00Z Disabled control of system clock
I20260807 15:07:00.918936 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34723
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:46883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:34723
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/testchainca.cert with env {}
W20260807 15:07:00.999172 27010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:00.999375 27010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:00.999396 27010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:01.000479 27010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:01.000514 27010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:01.000524 27010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:01.000535 27010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:01.000546 27010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:07:01.000557 27010 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:07:01.000566 27010 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 15:07:01.001971 27010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:34723
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:01.002171 27010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:01.002331 27010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:01.007174 27021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:01.007170 27017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:01.007166 27018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:01.007210 27010 server_base.cc:1034] running on GCE node
I20260807 15:07:01.007572 27010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:01.007848 27010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:01.008982 27010 hybrid_clock.cc:648] HybridClock initialized: now 1786115221008966 us; error 28 us; skew 500 ppm
I20260807 15:07:01.010322 27010 webserver.cc:460] Webserver started at http://127.18.129.62:40605/ using document root <none> and password file <none>
I20260807 15:07:01.010535 27010 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:01.010581 27010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:01.010706 27010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:01.011842 27010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "618eaf33175c4597b61b65705ae8ad6e"
format_stamp: "Formatted at 2026-08-07 15:07:01 on dist-test-slave-5bn2"
server_key: "d5af5c1a6d31f7ea05b557fb10f5aa0d"
server_key_iv: "f4530728ba6f4590beb08366a5252fb2"
server_key_version: "encryptionkey@0"
I20260807 15:07:01.012199 27010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "618eaf33175c4597b61b65705ae8ad6e"
format_stamp: "Formatted at 2026-08-07 15:07:01 on dist-test-slave-5bn2"
server_key: "d5af5c1a6d31f7ea05b557fb10f5aa0d"
server_key_iv: "f4530728ba6f4590beb08366a5252fb2"
server_key_version: "encryptionkey@0"
I20260807 15:07:01.013377 27010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:07:01.014021 27030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:01.014175 27010 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:01.014259 27010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "618eaf33175c4597b61b65705ae8ad6e"
format_stamp: "Formatted at 2026-08-07 15:07:01 on dist-test-slave-5bn2"
server_key: "d5af5c1a6d31f7ea05b557fb10f5aa0d"
server_key_iv: "f4530728ba6f4590beb08366a5252fb2"
server_key_version: "encryptionkey@0"
I20260807 15:07:01.014329 27010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:01.034989 27010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:01.035285 27010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:01.035418 27010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:01.039186 27010 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:34723
I20260807 15:07:01.039237 27105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:34723 every 8 connection(s)
I20260807 15:07:01.039546 27010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:01.040493 27106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:01.042781 27106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e: Bootstrap starting.
I20260807 15:07:01.043138 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 27010
I20260807 15:07:01.043227 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:01.043426 18948 external_mini_cluster.cc:1428] Setting key ff857630471bddc02f9f7dd13adf8027
I20260807 15:07:01.043433 27106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:01.043762 27106 log.cc:824] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:01.044476 27106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e: No bootstrap required, opened a new log
I20260807 15:07:01.046482 27106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618eaf33175c4597b61b65705ae8ad6e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34723 } }
I20260807 15:07:01.046662 27106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:01.046692 27106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 618eaf33175c4597b61b65705ae8ad6e, State: Initialized, Role: FOLLOWER
I20260807 15:07:01.046810 27106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618eaf33175c4597b61b65705ae8ad6e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34723 } }
I20260807 15:07:01.046921 27106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:01.046969 27106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:01.047020 27106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:01.047551 27106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618eaf33175c4597b61b65705ae8ad6e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34723 } }
I20260807 15:07:01.047672 27106 leader_election.cc:302] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 618eaf33175c4597b61b65705ae8ad6e; no voters:
I20260807 15:07:01.047837 27106 leader_election.cc:288] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:01.047995 27114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:01.048077 27106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:01.048156 27114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [term 1 LEADER]: Becoming Leader. State: Replica: 618eaf33175c4597b61b65705ae8ad6e, State: Running, Role: LEADER
I20260807 15:07:01.048246 27114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618eaf33175c4597b61b65705ae8ad6e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34723 } }
I20260807 15:07:01.048563 27114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 618eaf33175c4597b61b65705ae8ad6e. Latest consensus state: current_term: 1 leader_uuid: "618eaf33175c4597b61b65705ae8ad6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618eaf33175c4597b61b65705ae8ad6e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34723 } } }
I20260807 15:07:01.048640 27114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:01.048801 27113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "618eaf33175c4597b61b65705ae8ad6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618eaf33175c4597b61b65705ae8ad6e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34723 } } }
I20260807 15:07:01.048866 27113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:01.049165 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:01.043898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.18.129.62:34723)
0807 15:07:01.044089 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:01.044098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:01.044133 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:07:01.044617 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:01.044625 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:01.044684 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:01.044798 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:01.045027 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:01.045914 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:01.046987 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:01.047442 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:01.047531 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:01.048339 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:01.048343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:01.048343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:01.048544 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:01.048590 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:01.048606 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:01.048649 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:01.048791 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:07:01.048882 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260807 15:07:01.048974 27122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:01.049722 27122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:01.050145 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:07:01.051275 27122 catalog_manager.cc:1347] Generated new cluster ID: 4c5c2accdef1489cbb07b21122a8a11a
I20260807 15:07:01.051322 27122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:01.075773 27122 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:01.076316 27122 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:01.082796 27122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 618eaf33175c4597b61b65705ae8ad6e: Generated new TSK 0
I20260807 15:07:01.082995 27122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:07:01.139026 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:01.091393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38416 (local address 127.18.129.62:34723)
0807 15:07:01.091436 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:01.091438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:01.091506 (+ 68us) server_negotiation.cc:407] Connection header received
0807 15:07:01.091583 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:01.091584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:01.091593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:01.091643 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:07:01.091779 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:01.092275 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:01.093359 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:01.093541 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:01.093585 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:01.093757 (+ 172us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:07:01.138769 (+ 45012us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:07:01.138962 (+ 193us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38416: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:07:01.139101 27112 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38416: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 15:07:01.139205 27148 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:01.091295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34723 (local address 127.0.0.1:38416)
0807 15:07:01.091434 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:01.091445 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:01.091512 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:01.091655 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:01.091656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:01.091658 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:07:01.091723 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:01.091725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:01.092366 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:01.092368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:01.093284 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:01.093287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:01.093609 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:01.093609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:01.093668 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:01.093674 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:07:01.138939 (+ 45265us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:07:01.139088 (+ 149us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.129.62:34723: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260807 15:07:01.139551 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 27010
2026-08-07T15:07:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (255 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:07:01 dist-test-slave-5bn2 krb5kdc[27164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:07:01 dist-test-slave-5bn2 krb5kdc[27164](info): set up 2 sockets
Aug 07 15:07:01 dist-test-slave-5bn2 krb5kdc[27164](info): commencing operation
krb5kdc: starting...
W20260807 15:07:02.452713 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:07:02 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:07:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:07:02.567742 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37639
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:02.647270 27185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:02.647472 27185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:02.647502 27185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:02.648644 27185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:02.648684 27185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:02.648703 27185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:02.648720 27185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:02.648734 27185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:02.650187 27185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:02.650427 27185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:02.650619 27185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:02.655194 27193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:02.655246 27195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:02.655234 27192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:02.655387 27185 server_base.cc:1034] running on GCE node
I20260807 15:07:02.655625 27185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:02.655879 27185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:02.657015 27185 hybrid_clock.cc:648] HybridClock initialized: now 1786115222656997 us; error 36 us; skew 500 ppm
Aug 07 15:07:02 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115222, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:02.659482 27185 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:07:02.659901 27185 webserver.cc:460] Webserver started at http://127.18.129.62:42165/ using document root <none> and password file <none>
I20260807 15:07:02.660120 27185 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:02.660171 27185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:02.660297 27185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:02.661296 27185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "cc4dfcaf53fb422c824ed8c03a402608"
format_stamp: "Formatted at 2026-08-07 15:07:02 on dist-test-slave-5bn2"
server_key: "7ab2a8c9f3235e226abc85a3bdd92317"
server_key_iv: "b202ac2721c07da4cffe47bfff8cf9f9"
server_key_version: "encryptionkey@0"
I20260807 15:07:02.661639 27185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "cc4dfcaf53fb422c824ed8c03a402608"
format_stamp: "Formatted at 2026-08-07 15:07:02 on dist-test-slave-5bn2"
server_key: "7ab2a8c9f3235e226abc85a3bdd92317"
server_key_iv: "b202ac2721c07da4cffe47bfff8cf9f9"
server_key_version: "encryptionkey@0"
I20260807 15:07:02.662882 27185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:07:02.663587 27204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:02.663735 27185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:02.663820 27185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "cc4dfcaf53fb422c824ed8c03a402608"
format_stamp: "Formatted at 2026-08-07 15:07:02 on dist-test-slave-5bn2"
server_key: "7ab2a8c9f3235e226abc85a3bdd92317"
server_key_iv: "b202ac2721c07da4cffe47bfff8cf9f9"
server_key_version: "encryptionkey@0"
I20260807 15:07:02.663887 27185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:02.691093 27185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:02.693384 27185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:02.693533 27185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:02.697674 27185 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:37639
I20260807 15:07:02.697715 27273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:37639 every 8 connection(s)
I20260807 15:07:02.698058 27185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:02.698942 27274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:02.701171 27274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608: Bootstrap starting.
I20260807 15:07:02.701686 27274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:02.701936 27274 log.cc:824] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:02.702171 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 27185
I20260807 15:07:02.702263 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:02.702425 18948 external_mini_cluster.cc:1428] Setting key 509882e3d90974084096af8997f3093d
I20260807 15:07:02.702663 27274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608: No bootstrap required, opened a new log
I20260807 15:07:02.704728 27274 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4dfcaf53fb422c824ed8c03a402608" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37639 } }
I20260807 15:07:02.704901 27274 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:02.704936 27274 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4dfcaf53fb422c824ed8c03a402608, State: Initialized, Role: FOLLOWER
I20260807 15:07:02.705083 27274 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4dfcaf53fb422c824ed8c03a402608" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37639 } }
I20260807 15:07:02.705160 27274 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:02.705195 27274 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:02.705238 27274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:02.705883 27274 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4dfcaf53fb422c824ed8c03a402608" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37639 } }
I20260807 15:07:02.705989 27274 leader_election.cc:302] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc4dfcaf53fb422c824ed8c03a402608; no voters:
I20260807 15:07:02.706135 27274 leader_election.cc:288] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:02.706171 27279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:02.706327 27279 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [term 1 LEADER]: Becoming Leader. State: Replica: cc4dfcaf53fb422c824ed8c03a402608, State: Running, Role: LEADER
I20260807 15:07:02.706342 27274 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:02.706424 27279 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4dfcaf53fb422c824ed8c03a402608" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37639 } }
I20260807 15:07:02.707015 27281 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc4dfcaf53fb422c824ed8c03a402608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4dfcaf53fb422c824ed8c03a402608" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37639 } } }
I20260807 15:07:02.707111 27281 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:02.707268 27279 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc4dfcaf53fb422c824ed8c03a402608. Latest consensus state: current_term: 1 leader_uuid: "cc4dfcaf53fb422c824ed8c03a402608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc4dfcaf53fb422c824ed8c03a402608" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37639 } } }
I20260807 15:07:02.707331 27279 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:02.707459 27290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:02.708110 27290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:02.709697 27290 catalog_manager.cc:1347] Generated new cluster ID: c5e519d3c5414898979665c4a1cafe20
I20260807 15:07:02.709750 27290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:07:02 dist-test-slave-5bn2 krb5kdc[27164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:02.713692 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:02.702922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.18.129.62:37639)
0807 15:07:02.703116 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:02.703125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:02.703162 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:07:02.703977 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:02.703987 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:02.704050 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:02.704190 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:02.704754 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:02.706363 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:02.707498 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:02.708068 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:02.708170 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:02.710658 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:02.710662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:02.710663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:02.710674 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:02.712205 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:02.712522 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:02.712525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:02.712526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:02.712609 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:02.712782 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:02.712782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:02.712783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:02.713009 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:02.713154 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:02.713255 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 15:07:02.713381 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260807 15:07:02.722393 27290 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:02.722921 27290 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:02.730304 27290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc4dfcaf53fb422c824ed8c03a402608: Generated new TSK 0
I20260807 15:07:02.730455 27290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:07:02.759344 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37639
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:02.839673 27306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:02.839872 27306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:02.839895 27306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:02.841372 27306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:02.841428 27306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:02.841470 27306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:02.843575 27306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:02.843822 27306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:02.844080 27306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:02.849581 27319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:02.849645 27318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:02.849565 27323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:02.850891 27306 server_base.cc:1034] running on GCE node
I20260807 15:07:02.851163 27306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:02.851440 27306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:02.852599 27306 hybrid_clock.cc:648] HybridClock initialized: now 1786115222852584 us; error 32 us; skew 500 ppm
Aug 07 15:07:02 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115222, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:02.855110 27306 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:07:02.855556 27306 webserver.cc:460] Webserver started at http://127.18.129.1:44831/ using document root <none> and password file <none>
I20260807 15:07:02.855760 27306 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:02.855808 27306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:02.855934 27306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:02.856985 27306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "6a45c86732e84ad8bb5c93b16ba63d35"
format_stamp: "Formatted at 2026-08-07 15:07:02 on dist-test-slave-5bn2"
server_key: "c390b31ce2f067a6553bb74157063c3c"
server_key_iv: "a160dd1af84159cd633e3dffe51c1892"
server_key_version: "encryptionkey@0"
I20260807 15:07:02.857340 27306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "6a45c86732e84ad8bb5c93b16ba63d35"
format_stamp: "Formatted at 2026-08-07 15:07:02 on dist-test-slave-5bn2"
server_key: "c390b31ce2f067a6553bb74157063c3c"
server_key_iv: "a160dd1af84159cd633e3dffe51c1892"
server_key_version: "encryptionkey@0"
I20260807 15:07:02.858608 27306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:02.859349 27337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:02.859503 27306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:02.859592 27306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "6a45c86732e84ad8bb5c93b16ba63d35"
format_stamp: "Formatted at 2026-08-07 15:07:02 on dist-test-slave-5bn2"
server_key: "c390b31ce2f067a6553bb74157063c3c"
server_key_iv: "a160dd1af84159cd633e3dffe51c1892"
server_key_version: "encryptionkey@0"
I20260807 15:07:02.859663 27306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:02.881610 27306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:02.883386 27306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:02.883519 27306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:02.883741 27306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:02.884069 27306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:02.884106 27306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:02.884135 27306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:02.884157 27306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:02.889639 27306 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:39117
I20260807 15:07:02.889700 27493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:39117 every 8 connection(s)
I20260807 15:07:02.890004 27306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:07:02.894215 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 27306
I20260807 15:07:02.894340 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:07:02.894515 18948 external_mini_cluster.cc:1428] Setting key e9ba9936c8da4d8c7f119d6b7d2c1616
Aug 07 15:07:02 dist-test-slave-5bn2 krb5kdc[27164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115222, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:02.899726 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:02.890684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:40391 (local address 127.18.129.62:37639)
0807 15:07:02.890759 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:02.890761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:02.890988 (+ 227us) server_negotiation.cc:407] Connection header received
0807 15:07:02.891953 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:02.891955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:02.891967 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:02.892040 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:02.892685 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:02.893201 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:02.894262 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:02.894525 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:02.894610 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:02.897057 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:02.897058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:02.897059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:02.897067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:02.898416 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:02.898799 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:02.898800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:02.898801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:02.898891 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:02.899112 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:02.899113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:02.899113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:02.899245 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:02.899312 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:02.899539 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 15:07:02.899635 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:07:02.899905 27501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:02.890658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37639 (local address 127.18.129.1:40391)
0807 15:07:02.890836 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:02.890900 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:02.891809 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:02.892106 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:02.892110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:02.892363 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:02.892574 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:02.892579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:02.893274 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:02.893275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:02.894172 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:02.894175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:02.894633 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:02.894635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:02.894778 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:02.895366 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:02.895376 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:02.896949 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:02.898504 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:02.898510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:02.898512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:02.898731 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:02.898956 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:02.898957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:02.898957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:02.899060 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:02.899300 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:02.899303 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:02.899412 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 15:07:02.899484 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:07:02.899621 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260807 15:07:02.900353 27494 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37639
I20260807 15:07:02.900441 27494 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:02.900616 27494 heartbeater.cc:511] Master 127.18.129.62:37639 requested a full tablet report, sending...
I20260807 15:07:02.901103 27229 ts_manager.cc:194] Registered new tserver with Master: 6a45c86732e84ad8bb5c93b16ba63d35 (127.18.129.1:39117)
I20260807 15:07:02.901706 27229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:40391
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:07:02.919379 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37639
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:02.999555 27508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:02.999743 27508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:02.999765 27508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:03.000836 27508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:03.000869 27508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:03.000891 27508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:03.002286 27508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:03.002475 27508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:03.002696 27508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:03.007650 27521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:03.007678 27516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:03.007688 27518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:03.007946 27508 server_base.cc:1034] running on GCE node
I20260807 15:07:03.008152 27508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:03.008376 27508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:03.009512 27508 hybrid_clock.cc:648] HybridClock initialized: now 1786115223009501 us; error 30 us; skew 500 ppm
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.011941 27508 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:07:03.012334 27508 webserver.cc:460] Webserver started at http://127.18.129.2:44037/ using document root <none> and password file <none>
I20260807 15:07:03.012550 27508 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:03.012609 27508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:03.012730 27508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:03.013773 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "3fbb2accbf7841c9919a703861cf5c49"
format_stamp: "Formatted at 2026-08-07 15:07:03 on dist-test-slave-5bn2"
server_key: "8c89be5217bbd5c42243d708a81b6795"
server_key_iv: "5b734562dc49db88c3ef98a09b378fb7"
server_key_version: "encryptionkey@0"
I20260807 15:07:03.014103 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "3fbb2accbf7841c9919a703861cf5c49"
format_stamp: "Formatted at 2026-08-07 15:07:03 on dist-test-slave-5bn2"
server_key: "8c89be5217bbd5c42243d708a81b6795"
server_key_iv: "5b734562dc49db88c3ef98a09b378fb7"
server_key_version: "encryptionkey@0"
I20260807 15:07:03.015421 27508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:07:03.016093 27530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:03.016245 27508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:07:03.016320 27508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "3fbb2accbf7841c9919a703861cf5c49"
format_stamp: "Formatted at 2026-08-07 15:07:03 on dist-test-slave-5bn2"
server_key: "8c89be5217bbd5c42243d708a81b6795"
server_key_iv: "5b734562dc49db88c3ef98a09b378fb7"
server_key_version: "encryptionkey@0"
I20260807 15:07:03.016382 27508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:03.026582 27508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:03.028625 27508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:03.028755 27508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:03.028956 27508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:03.029229 27508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:03.029263 27508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:03.029292 27508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:03.029309 27508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:03.034808 27508 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:44225
I20260807 15:07:03.034868 27690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:44225 every 8 connection(s)
I20260807 15:07:03.035156 27508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:03.043650 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:03.035813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:54225 (local address 127.18.129.62:37639)
0807 15:07:03.035874 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:03.035876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:03.036054 (+ 178us) server_negotiation.cc:407] Connection header received
0807 15:07:03.036959 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:03.036960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:03.036973 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:03.037024 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:03.037587 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.038088 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.039095 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.039291 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.039347 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:03.041238 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:03.041240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:03.041240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:03.041248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:03.042359 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.042729 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.042731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.042731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.042789 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.043027 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.043028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.043029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.043151 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:03.043209 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:03.043461 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:07:03.043554 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:07:03.043794 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:03.035783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37639 (local address 127.18.129.2:54225)
0807 15:07:03.035942 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:03.035977 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:03.036854 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:03.037089 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:03.037093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:03.037322 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:03.037496 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:03.037500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.038168 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.038169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:03.039003 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:03.039006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.039359 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.039361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:03.039475 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:03.039901 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:03.039910 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:03.041146 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:03.042432 (+ 1286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:03.042439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:03.042441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:03.042662 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:03.042841 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:03.042842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:03.042842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:03.042969 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:03.043205 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:03.043212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:03.043342 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:07:03.043429 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:07:03.043587 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260807 15:07:03.043931 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 27508
I20260807 15:07:03.044040 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:07:03.044262 18948 external_mini_cluster.cc:1428] Setting key a6a394783d91ffee0869fd2282314dbf
I20260807 15:07:03.044304 27691 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37639
I20260807 15:07:03.044416 27691 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:03.044602 27691 heartbeater.cc:511] Master 127.18.129.62:37639 requested a full tablet report, sending...
I20260807 15:07:03.044956 27229 ts_manager.cc:194] Registered new tserver with Master: 3fbb2accbf7841c9919a703861cf5c49 (127.18.129.2:44225)
I20260807 15:07:03.045331 27229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:54225
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:07:03.067834 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37639
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:03.147197 27697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:03.147388 27697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:03.147418 27697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:03.148559 27697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:03.148600 27697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:03.148634 27697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:03.150065 27697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:03.150329 27697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:03.150538 27697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:03.155433 27703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:03.155489 27697 server_base.cc:1034] running on GCE node
W20260807 15:07:03.155534 27704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:03.155433 27708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:03.155848 27697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:03.156093 27697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:03.157263 27697 hybrid_clock.cc:648] HybridClock initialized: now 1786115223157238 us; error 34 us; skew 500 ppm
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.159906 27697 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:07:03.160337 27697 webserver.cc:460] Webserver started at http://127.18.129.3:35593/ using document root <none> and password file <none>
I20260807 15:07:03.160573 27697 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:03.160626 27697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:03.160760 27697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:03.161899 27697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "d937b088762943a5abbb8ac4483d64cd"
format_stamp: "Formatted at 2026-08-07 15:07:03 on dist-test-slave-5bn2"
server_key: "aa395a2a13ee1d97adb1aff2f47ec253"
server_key_iv: "0f70430fdc3b3f4ba5ac67b325b69f54"
server_key_version: "encryptionkey@0"
I20260807 15:07:03.162251 27697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "d937b088762943a5abbb8ac4483d64cd"
format_stamp: "Formatted at 2026-08-07 15:07:03 on dist-test-slave-5bn2"
server_key: "aa395a2a13ee1d97adb1aff2f47ec253"
server_key_iv: "0f70430fdc3b3f4ba5ac67b325b69f54"
server_key_version: "encryptionkey@0"
I20260807 15:07:03.163832 27697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:03.164558 27718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:03.164745 27697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:03.164844 27697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "d937b088762943a5abbb8ac4483d64cd"
format_stamp: "Formatted at 2026-08-07 15:07:03 on dist-test-slave-5bn2"
server_key: "aa395a2a13ee1d97adb1aff2f47ec253"
server_key_iv: "0f70430fdc3b3f4ba5ac67b325b69f54"
server_key_version: "encryptionkey@0"
I20260807 15:07:03.164906 27697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:03.186995 27697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:03.188714 27697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:03.188827 27697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:03.188997 27697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:03.189247 27697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:03.189276 27697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:03.189296 27697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:03.189308 27697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:03.194808 27697 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:46693
I20260807 15:07:03.194875 27900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:46693 every 8 connection(s)
I20260807 15:07:03.195189 27697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:03.202391 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 27697
I20260807 15:07:03.202484 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:07:03.202657 18948 external_mini_cluster.cc:1428] Setting key 8013700039c437bd879b85d8de54e879
I20260807 15:07:03.203959 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:03.195883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:41569 (local address 127.18.129.62:37639)
0807 15:07:03.195943 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:03.195945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:03.196144 (+ 199us) server_negotiation.cc:407] Connection header received
0807 15:07:03.197081 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:03.197083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:03.197096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:03.197147 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:03.197758 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.198263 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.199250 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.199455 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.199510 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:03.201455 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:03.201456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:03.201457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:03.201465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:03.202603 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.203057 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.203059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.203059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.203135 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.203354 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.203355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.203356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.203470 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:03.203527 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:03.203788 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 15:07:03.203869 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:07:03.204113 27905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:03.195847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37639 (local address 127.18.129.3:41569)
0807 15:07:03.196024 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:03.196060 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:03.196963 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:03.197218 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:03.197222 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:03.197468 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:03.197666 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:03.197670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.198345 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.198345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:03.199166 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:03.199168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.199525 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.199525 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:03.199654 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:03.200094 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:03.200102 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:03.201365 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:03.202670 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:03.202678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:03.202680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:03.202990 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:03.203189 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:03.203190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:03.203190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:03.203297 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:03.203526 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:03.203533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:03.203671 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 15:07:03.203756 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:07:03.203905 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260807 15:07:03.204548 27901 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37639
I20260807 15:07:03.204643 27901 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:03.204808 27901 heartbeater.cc:511] Master 127.18.129.62:37639 requested a full tablet report, sending...
I20260807 15:07:03.205149 27229 ts_manager.cc:194] Registered new tserver with Master: d937b088762943a5abbb8ac4483d64cd (127.18.129.3:46693)
I20260807 15:07:03.205502 27229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:41569
I20260807 15:07:03.214263 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/test-admin.keytab.
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.229585 18948 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 15:07:03.231261 27910 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:03.230472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.18.129.1:39117)
0807 15:07:03.230670 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:03.230675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:03.230686 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:07:03.230713 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:03.230715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:03.230730 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:03.230796 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:03.231187 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59270: BlockingRecv error: recv got EOF from 127.0.0.1:59270 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":115,"threads_started":1}
W20260807 15:07:03.231433 18948 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.232321 18948 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.233093 18948 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:07:03.240163 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:03.233547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.18.129.1:39117)
0807 15:07:03.233594 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:03.233596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:03.233625 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:07:03.233697 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:03.233698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:03.233708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:03.233752 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:03.234119 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.234842 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.235590 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.235897 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.235957 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:03.237819 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:03.237828 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:03.237828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:03.237835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:03.239143 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.239436 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.239439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.239439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.239492 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.239654 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.239656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.239656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.239782 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:03.239906 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:03.239982 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 15:07:03.240058 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 15:07:03.241822 27910 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:03.241274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.18.129.1:39117)
0807 15:07:03.241322 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:03.241323 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:03.241348 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:07:03.241397 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:03.241398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:03.241406 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:03.241450 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:03.241783 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59284: BlockingRecv error: recv got EOF from 127.0.0.1:59284 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 15:07:03.241904 18948 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.242729 18948 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.243475 18948 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:07:03.249153 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:03.243922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59294 (local address 127.18.129.1:39117)
0807 15:07:03.243960 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:03.243962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:03.243984 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:07:03.244029 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:03.244030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:03.244038 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:03.244075 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:03.244361 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.244836 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.245420 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.245607 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.245647 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:03.247325 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:03.247327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:03.247327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:03.247335 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:03.248359 (+ 1024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.248637 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.248637 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.248638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.248687 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.248806 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.248807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.248807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.248883 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:03.248943 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:03.249014 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:07:03.249073 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 15:07:03.250505 27910 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:03.250002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.18.129.1:39117)
0807 15:07:03.250058 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:03.250060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:03.250113 (+ 53us) server_negotiation.cc:407] Connection header received
0807 15:07:03.250172 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:03.250173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:03.250181 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:03.250232 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:03.250462 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59296: BlockingRecv error: recv got EOF from 127.0.0.1:59296 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 15:07:03.250559 18948 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115197351496-18948-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.251487 18948 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:03.252288 18948 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:07:03.258244 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:03.252715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59306 (local address 127.18.129.1:39117)
0807 15:07:03.252768 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:03.252770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:03.252794 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:07:03.252858 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:03.252859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:03.252867 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:03.252911 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:03.253254 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.253699 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.254339 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:03.254521 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:03.254596 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:03.256348 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:03.256350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:03.256350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:03.256356 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:03.257314 (+ 958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.257597 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.257598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.257599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.257647 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:03.257805 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:03.257806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:03.257806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:03.257891 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:03.257970 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:03.258085 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 15:07:03.258156 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:07:03.258769 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 27306
I20260807 15:07:03.269796 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 27508
I20260807 15:07:03.280799 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 27697
I20260807 15:07:03.290666 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 27185
2026-08-07T15:07:03Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1966 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27929](info): set up 2 sockets
Aug 07 15:07:03 dist-test-slave-5bn2 krb5kdc[27929](info): commencing operation
krb5kdc: starting...
W20260807 15:07:04.448549 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:07:04 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:07:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.62.keytab.
Entry for principal oryx/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.62.keytab.
I20260807 15:07:04.567771 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46499
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.62.keytab
--principal=oryx/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:04.647667 27950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:04.647861 27950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:04.647881 27950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:04.648974 27950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:04.649013 27950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:04.649025 27950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:04.649037 27950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:04.649047 27950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:04.650501 27950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.62.keytab
--principal=oryx/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:04.650700 27950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:04.650903 27950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:04.655628 27955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:04.655660 27956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:04.655619 27961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:04.655931 27950 server_base.cc:1034] running on GCE node
I20260807 15:07:04.656188 27950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:04.656445 27950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:04.657583 27950 hybrid_clock.cc:648] HybridClock initialized: now 1786115224657565 us; error 37 us; skew 500 ppm
Aug 07 15:07:04 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:04.660050 27950 init.cc:377] Logged in from keytab as oryx/127.18.129.62@KRBTEST.COM (short username oryx)
I20260807 15:07:04.660472 27950 webserver.cc:460] Webserver started at http://127.18.129.62:42293/ using document root <none> and password file <none>
I20260807 15:07:04.660670 27950 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:04.660709 27950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:04.660797 27950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:04.661767 27950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "0d381b52ca30418db8937a2848edd56a"
format_stamp: "Formatted at 2026-08-07 15:07:04 on dist-test-slave-5bn2"
server_key: "cb8d03b9bfc67b124aa793e8fead6419"
server_key_iv: "1017eea1a8b3688b17958d45501b3bd1"
server_key_version: "encryptionkey@0"
I20260807 15:07:04.662093 27950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "0d381b52ca30418db8937a2848edd56a"
format_stamp: "Formatted at 2026-08-07 15:07:04 on dist-test-slave-5bn2"
server_key: "cb8d03b9bfc67b124aa793e8fead6419"
server_key_iv: "1017eea1a8b3688b17958d45501b3bd1"
server_key_version: "encryptionkey@0"
I20260807 15:07:04.663340 27950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:04.664052 27976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:04.664214 27950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:04.664299 27950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "0d381b52ca30418db8937a2848edd56a"
format_stamp: "Formatted at 2026-08-07 15:07:04 on dist-test-slave-5bn2"
server_key: "cb8d03b9bfc67b124aa793e8fead6419"
server_key_iv: "1017eea1a8b3688b17958d45501b3bd1"
server_key_version: "encryptionkey@0"
I20260807 15:07:04.664364 27950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:04.676604 27950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:04.678429 27950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:04.678550 27950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:04.682492 27950 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:46499
I20260807 15:07:04.682552 28043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:46499 every 8 connection(s)
I20260807 15:07:04.682930 27950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:04.683936 28044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:04.686177 28044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: Bootstrap starting.
I20260807 15:07:04.686713 28044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:04.687011 28044 log.cc:824] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:04.687669 28044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: No bootstrap required, opened a new log
I20260807 15:07:04.689235 28044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } }
I20260807 15:07:04.689373 28044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:04.689406 28044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d381b52ca30418db8937a2848edd56a, State: Initialized, Role: FOLLOWER
I20260807 15:07:04.689532 28044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } }
I20260807 15:07:04.689591 28044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:04.689620 28044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:04.689668 28044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:04.690459 28044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } }
I20260807 15:07:04.690554 28044 leader_election.cc:302] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d381b52ca30418db8937a2848edd56a; no voters:
I20260807 15:07:04.690702 28044 leader_election.cc:288] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:04.690750 28048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:04.690918 28048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 1 LEADER]: Becoming Leader. State: Replica: 0d381b52ca30418db8937a2848edd56a, State: Running, Role: LEADER
I20260807 15:07:04.690968 28044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:04.691010 28048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } }
I20260807 15:07:04.691383 28050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d381b52ca30418db8937a2848edd56a. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } }
I20260807 15:07:04.691481 28050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:04.691356 28049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } }
I20260807 15:07:04.691658 28049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:04.691918 28061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:04.692416 28061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:04.692492 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 27950
I20260807 15:07:04.692589 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:04.692768 18948 external_mini_cluster.cc:1428] Setting key e1a7299395ec5138608db9c2d4874e33
I20260807 15:07:04.694608 28061 catalog_manager.cc:1347] Generated new cluster ID: 37779a6cc96443ad963ac6ff683842dd
I20260807 15:07:04.694666 28061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:07:04 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:04.702181 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:04.693213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.18.129.62:46499)
0807 15:07:04.693791 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:04.693800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:04.693832 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:07:04.693942 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:04.693949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:04.694003 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:04.694152 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:04.694791 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:04.695748 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:04.696516 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:04.696942 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:04.697043 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:04.699192 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:04.699196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:04.699196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:04.699208 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:04.700682 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:04.701001 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:04.701004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:04.701004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:04.701104 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:04.701278 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:04.701279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:04.701279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:04.701523 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:04.701666 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:04.701772 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 15:07:04.702005 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":73,"threads_started":1}
I20260807 15:07:04.707566 28061 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:04.708068 28061 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:04.711161 28061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: Generated new TSK 0
I20260807 15:07:04.711323 28061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab.
Entry for principal oryx/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab.
I20260807 15:07:04.747925 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46499
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab
--principal=oryx/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:04.847580 28088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:04.847788 28088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:04.847829 28088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:04.848966 28088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:04.849007 28088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:04.849041 28088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:04.850888 28088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab
--principal=oryx/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:04.851159 28088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:04.851436 28088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:04.856710 28100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:04.856702 28096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:04.856762 28088 server_base.cc:1034] running on GCE node
W20260807 15:07:04.856686 28094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:04.857090 28088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:04.857345 28088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:04.858510 28088 hybrid_clock.cc:648] HybridClock initialized: now 1786115224858486 us; error 36 us; skew 500 ppm
Aug 07 15:07:04 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:04.861202 28088 init.cc:377] Logged in from keytab as oryx/127.18.129.1@KRBTEST.COM (short username oryx)
I20260807 15:07:04.861660 28088 webserver.cc:460] Webserver started at http://127.18.129.1:42085/ using document root <none> and password file <none>
I20260807 15:07:04.861909 28088 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:04.861961 28088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:04.862085 28088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:04.863183 28088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "4ff6d3654b92431fb6c63e8f63b41a2d"
format_stamp: "Formatted at 2026-08-07 15:07:04 on dist-test-slave-5bn2"
server_key: "a2bcbe60daaff829d8b950b0d6502a66"
server_key_iv: "cc5fee8841d2e16123b7c6365fa2f0de"
server_key_version: "encryptionkey@0"
I20260807 15:07:04.863546 28088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "4ff6d3654b92431fb6c63e8f63b41a2d"
format_stamp: "Formatted at 2026-08-07 15:07:04 on dist-test-slave-5bn2"
server_key: "a2bcbe60daaff829d8b950b0d6502a66"
server_key_iv: "cc5fee8841d2e16123b7c6365fa2f0de"
server_key_version: "encryptionkey@0"
I20260807 15:07:04.864801 28088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 15:07:04.865578 28109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:04.865778 28088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:07:04.865890 28088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "4ff6d3654b92431fb6c63e8f63b41a2d"
format_stamp: "Formatted at 2026-08-07 15:07:04 on dist-test-slave-5bn2"
server_key: "a2bcbe60daaff829d8b950b0d6502a66"
server_key_iv: "cc5fee8841d2e16123b7c6365fa2f0de"
server_key_version: "encryptionkey@0"
I20260807 15:07:04.865960 28088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:04.884959 28088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:04.886830 28088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:04.887018 28088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:04.887261 28088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:04.887591 28088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:04.887630 28088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:04.887665 28088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:04.887689 28088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:04.894232 28088 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:36789
I20260807 15:07:04.894294 28274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:36789 every 8 connection(s)
I20260807 15:07:04.894632 28088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:07:04 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:04.903530 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:04.895403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:44433 (local address 127.18.129.62:46499)
0807 15:07:04.895490 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:04.895493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:04.895654 (+ 161us) server_negotiation.cc:407] Connection header received
0807 15:07:04.896163 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:04.896164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:04.896179 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:04.896230 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:04.897147 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:04.897644 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:04.898673 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:04.898887 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:04.898943 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:04.900978 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:04.900980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:04.900980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:04.900988 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:04.902256 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:04.902665 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:04.902666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:04.902666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:04.902717 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:04.902994 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:04.902994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:04.902994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:04.903105 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:04.903156 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:04.903397 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 15:07:04.903465 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:07:04.903703 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:04.895359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.1:44433)
0807 15:07:04.895524 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:04.895564 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:04.896077 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:04.896285 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:04.896289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:04.896877 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:04.897071 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:04.897076 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:04.897733 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:04.897734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:04.898596 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:04.898599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:04.898953 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:04.898954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:04.899059 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:04.899494 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:04.899502 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:04.900887 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:04.902360 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:04.902368 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:04.902370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:04.902597 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:04.902769 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:04.902770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:04.902770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:04.902936 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:04.903158 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:04.903162 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:04.903284 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 15:07:04.903372 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:07:04.903497 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260807 15:07:04.904054 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 28088
I20260807 15:07:04.904165 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:07:04.904265 28275 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46499
I20260807 15:07:04.904354 28275 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:04.904381 18948 external_mini_cluster.cc:1428] Setting key 8896944af085d203f2937a9afc7a004c
I20260807 15:07:04.904525 28275 heartbeater.cc:511] Master 127.18.129.62:46499 requested a full tablet report, sending...
I20260807 15:07:04.905019 28002 ts_manager.cc:194] Registered new tserver with Master: 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789)
I20260807 15:07:04.905797 28002 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:44433
WARNING: no policy specified for oryx/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab.
Entry for principal oryx/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab.
I20260807 15:07:04.929658 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46499
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab
--principal=oryx/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:05.011502 28285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:05.011722 28285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:05.011747 28285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:05.012838 28285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:05.012876 28285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:05.012902 28285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:05.014360 28285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab
--principal=oryx/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:05.014556 28285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:05.014747 28285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:05.019960 28291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:05.019958 28292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.020037 28285 server_base.cc:1034] running on GCE node
W20260807 15:07:05.019971 28294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.020331 28285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:05.020545 28285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:05.021675 28285 hybrid_clock.cc:648] HybridClock initialized: now 1786115225021666 us; error 26 us; skew 500 ppm
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:05.024142 28285 init.cc:377] Logged in from keytab as oryx/127.18.129.2@KRBTEST.COM (short username oryx)
I20260807 15:07:05.024564 28285 webserver.cc:460] Webserver started at http://127.18.129.2:38405/ using document root <none> and password file <none>
I20260807 15:07:05.024744 28285 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.024783 28285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.024873 28285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:05.025863 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "1ca0652f5f53478ea3dd9ac1fd968025"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "135bb3dbfffd748c119cae316140e6af"
server_key_iv: "d2e4e18e644a3d9fd2bb401b2645296f"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.026171 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "1ca0652f5f53478ea3dd9ac1fd968025"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "135bb3dbfffd748c119cae316140e6af"
server_key_iv: "d2e4e18e644a3d9fd2bb401b2645296f"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.027390 28285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:05.028035 28301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.028206 28285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:05.028291 28285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "1ca0652f5f53478ea3dd9ac1fd968025"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "135bb3dbfffd748c119cae316140e6af"
server_key_iv: "d2e4e18e644a3d9fd2bb401b2645296f"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.028347 28285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.039904 28285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:05.041706 28285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:05.041818 28285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:05.042011 28285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:05.042301 28285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:05.042335 28285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.042357 28285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:05.042374 28285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.048035 28285 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43269
I20260807 15:07:05.048120 28485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43269 every 8 connection(s)
I20260807 15:07:05.048408 28285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:05.054876 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 28285
I20260807 15:07:05.054965 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:07:05.055153 18948 external_mini_cluster.cc:1428] Setting key 397199f1d5d75ea63bb6841b4b6acc85
I20260807 15:07:05.057446 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.049076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:52219 (local address 127.18.129.62:46499)
0807 15:07:05.049125 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.049127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.049326 (+ 199us) server_negotiation.cc:407] Connection header received
0807 15:07:05.049803 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.049804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.049814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.049872 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.050794 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.051348 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.052319 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.052567 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.052629 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.054637 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.054639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.054639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.054646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.056158 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.056598 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.056598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.056599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.056652 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.056894 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.056894 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.056894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.056989 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.057033 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.057309 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.057379 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:07:05.057689 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.049031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.2:52219)
0807 15:07:05.049204 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.049246 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.049727 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.049923 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.049926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.050494 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.050700 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.050705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.051417 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.051418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.052220 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.052222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.052637 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.052638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.052743 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.053216 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.053224 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.054525 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.056230 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.056238 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.056240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.056514 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.056705 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.056706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.056707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.056841 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.057037 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.057042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.057192 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.057299 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.057452 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260807 15:07:05.058224 28486 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46499
I20260807 15:07:05.058326 28486 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:05.058509 28486 heartbeater.cc:511] Master 127.18.129.62:46499 requested a full tablet report, sending...
I20260807 15:07:05.058872 28002 ts_manager.cc:194] Registered new tserver with Master: 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269)
I20260807 15:07:05.059249 28002 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:52219
WARNING: no policy specified for oryx/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab.
Entry for principal oryx/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab.
I20260807 15:07:05.079435 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46499
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab
--principal=oryx/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:05.159444 28492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:05.159646 28492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:05.159668 28492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:05.160738 28492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:05.160770 28492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:05.160794 28492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:05.162179 28492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab
--principal=oryx/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:05.162370 28492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:05.162541 28492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:05.167407 28504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:05.167404 28499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.167518 28492 server_base.cc:1034] running on GCE node
W20260807 15:07:05.167752 28501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.167960 28492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:05.168169 28492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:05.169294 28492 hybrid_clock.cc:648] HybridClock initialized: now 1786115225169283 us; error 27 us; skew 500 ppm
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:05.171772 28492 init.cc:377] Logged in from keytab as oryx/127.18.129.3@KRBTEST.COM (short username oryx)
I20260807 15:07:05.172158 28492 webserver.cc:460] Webserver started at http://127.18.129.3:43671/ using document root <none> and password file <none>
I20260807 15:07:05.172348 28492 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.172386 28492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.172477 28492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:05.173445 28492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "7d97875043184639b13b66f88a153466"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "cf09ad468b418b317e4a651e179c2c03"
server_key_iv: "95ccf32c2dff00c3fcd8aa84045d1573"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.173794 28492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "7d97875043184639b13b66f88a153466"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "cf09ad468b418b317e4a651e179c2c03"
server_key_iv: "95ccf32c2dff00c3fcd8aa84045d1573"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.174994 28492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:05.175690 28513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.175849 28492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.175940 28492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "7d97875043184639b13b66f88a153466"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "cf09ad468b418b317e4a651e179c2c03"
server_key_iv: "95ccf32c2dff00c3fcd8aa84045d1573"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.175999 28492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.194000 28492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:05.196139 28492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:05.196272 28492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:05.196472 28492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:05.196756 28492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:05.196790 28492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.196820 28492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:05.196843 28492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.202428 28492 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:46147
I20260807 15:07:05.202477 28690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:46147 every 8 connection(s)
I20260807 15:07:05.202780 28492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:07:05.204236 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 28492
I20260807 15:07:05.204335 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:07:05.204514 18948 external_mini_cluster.cc:1428] Setting key e523876ca16ba11b54604f343db60629
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:05.211347 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.203470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:55555 (local address 127.18.129.62:46499)
0807 15:07:05.203527 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.203529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.203696 (+ 167us) server_negotiation.cc:407] Connection header received
0807 15:07:05.204230 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.204232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.204241 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.204298 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.205424 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.205912 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.206841 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.207064 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.207118 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.209099 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.209100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.209101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.209107 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.210178 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.210508 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.210509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.210509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.210553 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.210729 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.210729 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.210729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.210821 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.210914 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.211193 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.211280 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:07:05.211515 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.203433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.3:55555)
0807 15:07:05.203580 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.203620 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.204122 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.204377 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.204382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.205047 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.205310 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.205317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.205978 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.205978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.206780 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.206782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.207133 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.207134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.207242 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.207700 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.207708 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.209005 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.210241 (+ 1236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.210247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.210248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.210452 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.210599 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.210599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.210599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.210688 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.210904 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.210908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.211042 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.211149 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.211300 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260807 15:07:05.211961 28692 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46499
I20260807 15:07:05.212042 28692 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:05.212181 28692 heartbeater.cc:511] Master 127.18.129.62:46499 requested a full tablet report, sending...
I20260807 15:07:05.212493 28002 ts_manager.cc:194] Registered new tserver with Master: 7d97875043184639b13b66f88a153466 (127.18.129.3:46147)
I20260807 15:07:05.212860 28002 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:55555
I20260807 15:07:05.216075 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.129.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.61.keytab.
Entry for principal oryx/127.18.129.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.61.keytab.
I20260807 15:07:05.239656 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.61:35837
--webserver_interface=127.18.129.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.61.keytab
--principal=oryx/127.18.129.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46499,127.18.129.61:35837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:05.320889 28697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:05.321089 28697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:05.321108 28697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:05.322220 28697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:05.322259 28697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:05.322273 28697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:05.322295 28697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:05.322319 28697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:05.323760 28697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46499,127.18.129.61:35837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.61.keytab
--principal=oryx/127.18.129.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.61:35837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.129.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
W20260807 15:07:05.323868 28697 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.129.62:46499,127.18.129.61:35837'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 15:07:05.324007 28697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:05.324209 28697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:05.328948 28708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:05.328962 28703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:05.328964 28704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.329120 28697 server_base.cc:1034] running on GCE node
I20260807 15:07:05.329360 28697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:05.329633 28697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:05.330773 28697 hybrid_clock.cc:648] HybridClock initialized: now 1786115225330759 us; error 28 us; skew 500 ppm
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:05.333240 28697 init.cc:377] Logged in from keytab as oryx/127.18.129.61@KRBTEST.COM (short username oryx)
I20260807 15:07:05.333715 28697 webserver.cc:460] Webserver started at http://127.18.129.61:37587/ using document root <none> and password file <none>
I20260807 15:07:05.333916 28697 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.333963 28697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.334088 28697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:05.335175 28697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data/instance:
uuid: "76d63311c9c9410c840cca74d203c16b"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "074f9034a48e33dfc36ac48e45170cc6"
server_key_iv: "0fc0f6dccb72c39b2986dd4e7f6ae962"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.335512 28697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal/instance:
uuid: "76d63311c9c9410c840cca74d203c16b"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "074f9034a48e33dfc36ac48e45170cc6"
server_key_iv: "0fc0f6dccb72c39b2986dd4e7f6ae962"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.336829 28697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:07:05.337529 28719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.337699 28697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.337786 28697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
uuid: "76d63311c9c9410c840cca74d203c16b"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "074f9034a48e33dfc36ac48e45170cc6"
server_key_iv: "0fc0f6dccb72c39b2986dd4e7f6ae962"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.337846 28697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.344748 28697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:05.346756 28697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:05.346916 28697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:05.351323 28697 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.61:35837
I20260807 15:07:05.351385 28796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.61:35837 every 8 connection(s)
I20260807 15:07:05.351697 28697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
I20260807 15:07:05.352839 28797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:05.354168 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 28697
I20260807 15:07:05.355098 28797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.61@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.129.61@KRBTEST.COM
I20260807 15:07:05.365033 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.355880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34532 (local address 127.18.129.62:46499)
0807 15:07:05.356041 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.356044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.356419 (+ 375us) server_negotiation.cc:407] Connection header received
0807 15:07:05.356458 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.356459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.356498 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.356581 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.357279 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.358020 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.359357 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.359665 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.359731 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.362058 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.362060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.362060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.362069 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.363527 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.363939 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.363941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.363941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.364006 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.364267 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.364268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.364268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.364456 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.364532 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.364853 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.364942 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:07:05.365299 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.355793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:34532)
0807 15:07:05.356252 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.356315 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.356404 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.356649 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.356652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.357016 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.357187 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.357192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.358128 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.358129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.359282 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.359285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.359747 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.359748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.359835 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.360332 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.360342 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.361964 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.363604 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.363611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.363613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.363866 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.364075 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.364076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.364076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.364200 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.364527 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.364533 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.364677 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.364778 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.364895 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":4,"thread_start_us":90,"threads_started":1}
I20260807 15:07:05.365389 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.354831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.18.129.61:35837)
0807 15:07:05.355088 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.355098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.355214 (+ 116us) server_negotiation.cc:407] Connection header received
0807 15:07:05.355905 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.355915 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.355980 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.356128 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.357019 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.358275 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.359331 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.359809 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.359860 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.362467 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.362476 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.362476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.362484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.364099 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.364471 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.364474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.364474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.364537 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.364752 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.364754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.364754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.364948 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.365099 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.365197 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.365266 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
I20260807 15:07:05.365936 28797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.61@KRBTEST.COM for oryx/127.18.129.61@KRBTEST.COM
I20260807 15:07:05.371433 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.366164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.0.0.1:37668)
0807 15:07:05.366214 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.366224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.366257 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.366340 (+ 83us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.366341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.366445 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.366473 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.366475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.367158 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.367158 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.367632 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.367633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.367915 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.367915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.367980 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.368369 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.368372 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.369412 (+ 1040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.370727 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.370727 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.370727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.370890 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.371028 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.371028 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.371028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.371076 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.371223 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.371224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.371253 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.371295 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.371362 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:07:05.371485 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.366207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37668 (local address 127.18.129.61:35837)
0807 15:07:05.366236 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.366237 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.366247 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:07:05.366293 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.366294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.366301 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.366336 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.366517 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.367099 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.367686 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.367866 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.367910 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.369492 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.369493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.369493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.369499 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.370670 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.370939 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.370939 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.370940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.370984 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.371108 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.371108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.371109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.371186 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.371227 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.371311 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.371403 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":13}
I20260807 15:07:05.372711 28797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Bootstrap starting.
I20260807 15:07:05.373541 28797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:05.373795 28797 log.cc:824] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:05.374454 28797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: No bootstrap required, opened a new log
I20260807 15:07:05.375849 28797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } }
I20260807 15:07:05.375976 28797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:05.376010 28797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d63311c9c9410c840cca74d203c16b, State: Initialized, Role: FOLLOWER
I20260807 15:07:05.376127 28797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } }
I20260807 15:07:05.376341 28811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } } }
I20260807 15:07:05.376391 28797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:05.376430 28811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:07:05.378530 28827 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:07:05.378582 28827 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:07:05.380318 28697 master_runner.cc:428] Detected that this master 76d63311c9c9410c840cca74d203c16b is joining an existing cluster
I20260807 15:07:05.380407 28697 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.129.61:35837
I20260807 15:07:05.387796 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.381417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.18.129.62:46499)
0807 15:07:05.381472 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.381474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.381905 (+ 431us) server_negotiation.cc:407] Connection header received
0807 15:07:05.381937 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.381938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.381949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.382000 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.382324 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.382826 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.383845 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.384044 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.384097 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.385093 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.385095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.385095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.385103 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.386537 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.386926 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.386928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.386928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.387000 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.387224 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.387225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.387225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.387356 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.387438 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.387610 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.387696 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:07:05.388109 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.381618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:34536)
0807 15:07:05.381800 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.381812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.381883 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.382008 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.382008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.382192 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.382256 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.382257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.382944 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.382945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.383790 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.383792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.384118 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.384118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.384178 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.384653 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.384660 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.385007 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.386597 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.386598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.386599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.386812 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.387070 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.387070 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.387071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.387158 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.387433 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.387434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.387476 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.387890 (+ 414us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.387991 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260807 15:07:05.390349 28838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.383130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.0.0.1:37672)
0807 15:07:05.383264 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.383274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.383331 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.383451 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.383452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.383645 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.383706 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.383707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.384602 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.384603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.385470 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.385473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.386162 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.386163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.386243 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.387048 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.387053 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.387435 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.389196 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.389197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.389198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.389382 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.389692 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.389693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.389693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.389753 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.389985 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.389986 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.390045 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.390127 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.390228 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260807 15:07:05.390424 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.383093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37672 (local address 127.18.129.61:35837)
0807 15:07:05.383129 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.383130 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.383335 (+ 205us) server_negotiation.cc:407] Connection header received
0807 15:07:05.383378 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.383379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.383388 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.383439 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.383770 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.384455 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.385590 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.386085 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.386153 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.387490 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.387491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.387491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.387497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.389122 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.389561 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.389562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.389563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.389625 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.389803 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.389803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.389803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.389923 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.389983 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.390149 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.390290 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":18}
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.62@KRBTEST.COM for oryx/127.18.129.61@KRBTEST.COM
I20260807 15:07:05.398144 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.390290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.0.0.1:37674)
0807 15:07:05.390963 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.391002 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.391073 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.391316 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.391322 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.391573 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.391616 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.391618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.392565 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.392566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.393118 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.393120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.393601 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.393602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.393689 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.394139 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.394146 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.395599 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.397209 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.397216 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.397218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.397411 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.397613 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.397613 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.397614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.397662 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.397882 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.397886 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.397924 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.397979 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.398062 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":54,"threads_started":1}
I20260807 15:07:05.398214 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.390595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.18.129.61:35837)
0807 15:07:05.390726 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.390728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.391051 (+ 323us) server_negotiation.cc:407] Connection header received
0807 15:07:05.391195 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.391196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.391207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.391310 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.391682 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.392498 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.393186 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.393486 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.393545 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.395676 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.395677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.395678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.395685 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.397137 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.397475 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.397476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.397476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.397553 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.397707 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.397708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.397708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.397828 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.397885 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.398023 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.398111 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260807 15:07:05.398924 28002 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.129.61:35837: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "76d63311c9c9410c840cca74d203c16b"
member_type: NON_VOTER
last_known_addr {
host: "127.18.129.61"
port: 35837
}
attrs {
promote: true
}
}
dest_uuid: "0d381b52ca30418db8937a2848edd56a"
cas_config_opid_index: -1
I20260807 15:07:05.399278 28002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } }
I20260807 15:07:05.399751 28847 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 76d63311c9c9410c840cca74d203c16b (127.18.129.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } }
I20260807 15:07:05.400063 28760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:05.400610 28847 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.129.61:35837
I20260807 15:07:05.400822 28760 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Refusing update from remote peer 0d381b52ca30418db8937a2848edd56a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 15:07:05.400787 28843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } }
I20260807 15:07:05.400868 28843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:05.400899 28697 master.cc:537] Master@127.18.129.61:35837 shutting down...
I20260807 15:07:05.401139 28843 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Connected to new peer: Peer: permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:07:05.401829 27982 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 15:07:05.402241 27982 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a -> Peer 76d63311c9c9410c840cca74d203c16b (127.18.129.61:35837): Couldn't send request to peer 76d63311c9c9410c840cca74d203c16b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:07:05.403528 28697 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:05.403620 28697 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:05.403671 28697 tablet_replica.cc:330] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: stopping tablet replica
I20260807 15:07:05.416417 28697 master.cc:552] Master@127.18.129.61:35837 shutdown complete.
I20260807 15:07:05.416469 28697 master_runner.cc:305] Clearing existing system tablet
I20260807 15:07:05.416599 28697 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.416649 28697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.417207 28697 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260807 15:07:05.417618 28854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.417737 28697 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.417811 28697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.417837 28697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
uuid: "76d63311c9c9410c840cca74d203c16b"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "074f9034a48e33dfc36ac48e45170cc6"
server_key_iv: "0fc0f6dccb72c39b2986dd4e7f6ae962"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.418633 28697 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:05.420285 28697 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:07:05.420358 28697 log.cc:1196] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 15:07:05.420548 28697 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Deleting consensus metadata
I20260807 15:07:05.420679 28697 master_runner.cc:315] Copying system tablet from 127.18.129.62:46499
I20260807 15:07:05.421268 28697 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: tablet copy: Beginning tablet copy session from remote peer at address 127.18.129.62:46499
I20260807 15:07:05.427385 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.421730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.18.129.62:46499)
0807 15:07:05.421790 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.421792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.421828 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:07:05.421913 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.421915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.421924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.421979 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.422369 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.422892 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.423938 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.424124 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.424165 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.425116 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.425117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.425117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.425124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.426363 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.426729 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.426729 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.426730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.426776 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.426965 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.426965 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.426966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.427075 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.427127 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.427264 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.427323 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:07:05.427456 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.421636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:34546)
0807 15:07:05.421766 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.421777 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.421853 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.421997 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.421998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.422204 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.422281 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.422284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.422965 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.422966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.423861 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.423864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.424181 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.424182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.424251 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.424661 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.424667 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.425041 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.426435 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.426435 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.426436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.426665 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.426826 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.426827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.426827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.426912 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.427131 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.427132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.427180 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.427248 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.427350 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260807 15:07:05.427832 28027 tablet_copy_service.cc:140] P 0d381b52ca30418db8937a2848edd56a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 76d63311c9c9410c840cca74d203c16b ({username='oryx', principal='oryx/127.18.129.61@KRBTEST.COM'} at 127.0.0.1:34546)
I20260807 15:07:05.427944 28027 tablet_copy_service.cc:161] P 0d381b52ca30418db8937a2848edd56a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 76d63311c9c9410c840cca74d203c16b at {username='oryx', principal='oryx/127.18.129.61@KRBTEST.COM'} at 127.0.0.1:34546: session id = 76d63311c9c9410c840cca74d203c16b-00000000000000000000000000000000
I20260807 15:07:05.428615 28027 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:07:05.429173 28697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:07:05.430449 28697 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: tablet copy: Starting download of 0 data blocks...
I20260807 15:07:05.430604 28697 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: tablet copy: Starting download of 1 WAL segments...
I20260807 15:07:05.431558 28697 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:07:05.432420 28027 tablet_copy_service.cc:342] P 0d381b52ca30418db8937a2848edd56a: Request end of tablet copy session 76d63311c9c9410c840cca74d203c16b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.129.61@KRBTEST.COM'} at 127.0.0.1:34546
I20260807 15:07:05.432477 28027 tablet_copy_service.cc:433] P 0d381b52ca30418db8937a2848edd56a: ending tablet copy session 76d63311c9c9410c840cca74d203c16b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 76d63311c9c9410c840cca74d203c16b
I20260807 15:07:05.434644 28697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:05.435865 28867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:05.435974 28862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.436048 28697 server_base.cc:1034] running on GCE node
W20260807 15:07:05.435985 28864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.436264 28697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:05.436379 28697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:05.437486 28697 hybrid_clock.cc:648] HybridClock initialized: now 1786115225437479 us; error 24 us; skew 500 ppm
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:05.438766 28697 init.cc:377] Logged in from keytab as oryx/127.18.129.61@KRBTEST.COM (short username oryx)
I20260807 15:07:05.439029 28697 webserver.cc:460] Webserver started at http://127.18.129.61:33467/ using document root <none> and password file <none>
I20260807 15:07:05.439144 28697 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.439177 28697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.439708 28697 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:07:05.440174 28876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.440300 28697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.440344 28697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
uuid: "76d63311c9c9410c840cca74d203c16b"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "074f9034a48e33dfc36ac48e45170cc6"
server_key_iv: "0fc0f6dccb72c39b2986dd4e7f6ae962"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.440400 28697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.445807 28697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:05.447184 28697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:05.450642 28697 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.61:35837
I20260807 15:07:05.451026 28962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.61:35837 every 8 connection(s)
I20260807 15:07:05.451067 28697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
I20260807 15:07:05.451750 28964 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:07:05.452154 28964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Bootstrap starting.
I20260807 15:07:05.454780 28964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:07:05.455054 28964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Bootstrap complete.
I20260807 15:07:05.455272 28964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } }
I20260807 15:07:05.455367 28964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 76d63311c9c9410c840cca74d203c16b, State: Initialized, Role: LEARNER
I20260807 15:07:05.455416 28964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } }
I20260807 15:07:05.455597 28964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:05.455600 28968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } }
I20260807 15:07:05.455727 28968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: This master's current role is: LEARNER
I20260807 15:07:05.457118 28982 catalog_manager.cc:1259] Loaded cluster ID: 37779a6cc96443ad963ac6ff683842dd
I20260807 15:07:05.457167 28982 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: loading cluster ID for follower catalog manager: success
I20260807 15:07:05.458108 28982 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: acquiring CA information for follower catalog manager: success
I20260807 15:07:05.458357 28982 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:07:05.473893 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.467010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.18.129.61:35837)
0807 15:07:05.467117 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.467119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.467133 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:07:05.467159 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.467160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.467169 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.467257 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.467963 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.468654 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.469734 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.469934 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.469983 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.471407 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.471409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.471409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.471415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.472769 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.473153 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.473154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.473154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.473229 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.473433 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.473433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.473433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.473525 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.473603 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.473740 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.473804 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
WARNING: no policy specified for oryx/127.18.129.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.60.keytab.
Entry for principal oryx/127.18.129.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.60.keytab.
I20260807 15:07:05.501638 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.60:44647
--webserver_interface=127.18.129.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.60.keytab
--principal=oryx/127.18.129.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46499,127.18.129.61:35837,127.18.129.60:44647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:05.585076 28988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:05.585286 28988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:05.585314 28988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:05.586432 28988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:05.586472 28988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:05.586484 28988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:05.586506 28988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:05.586529 28988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:05.588068 28988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46499,127.18.129.61:35837,127.18.129.60:44647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.60.keytab
--principal=oryx/127.18.129.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.60:44647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.129.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:05.588299 28988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:05.588505 28988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:05.593413 28994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.593472 28988 server_base.cc:1034] running on GCE node
W20260807 15:07:05.593425 28995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:05.593428 28997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.593816 28988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:05.594055 28988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:05.595187 28988 hybrid_clock.cc:648] HybridClock initialized: now 1786115225595174 us; error 28 us; skew 500 ppm
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:05.597529 28988 init.cc:377] Logged in from keytab as oryx/127.18.129.60@KRBTEST.COM (short username oryx)
I20260807 15:07:05.597944 28988 webserver.cc:460] Webserver started at http://127.18.129.60:39817/ using document root <none> and password file <none>
I20260807 15:07:05.598143 28988 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.598192 28988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.598316 28988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:05.599367 28988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data/instance:
uuid: "2a8d99b921f5477889667008c53b93ed"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "f7a1da6a76a3281f144d39022918eebd"
server_key_iv: "80b19cff0a511fd1f634d727f8e95851"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.599704 28988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal/instance:
uuid: "2a8d99b921f5477889667008c53b93ed"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "f7a1da6a76a3281f144d39022918eebd"
server_key_iv: "80b19cff0a511fd1f634d727f8e95851"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.600916 28988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:07:05.601545 29008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.601691 28988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:05.601776 28988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
uuid: "2a8d99b921f5477889667008c53b93ed"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "f7a1da6a76a3281f144d39022918eebd"
server_key_iv: "80b19cff0a511fd1f634d727f8e95851"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.601833 28988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.635779 28988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:05.637586 28988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:05.637725 28988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:05.641719 28988 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.60:44647
I20260807 15:07:05.641767 29085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.60:44647 every 8 connection(s)
I20260807 15:07:05.642091 28988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
I20260807 15:07:05.643106 29086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:05.644675 29086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:07:05.647738 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 28988
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.60@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.129.60@KRBTEST.COM
I20260807 15:07:05.653553 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.645510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.18.129.62:46499)
0807 15:07:05.645565 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.645567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.645596 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:07:05.646037 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.646038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.646048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.646103 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.646703 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.647222 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.648400 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.648689 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.648749 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.650827 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.650829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.650829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.650837 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.652193 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.652615 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.652617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.652617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.652688 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.652903 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.652904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.652904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.653045 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.653114 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.653363 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.653440 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:07:05.653980 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.645309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:34548)
0807 15:07:05.645486 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.645527 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.645962 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.646170 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.646173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.646424 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.646639 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.646643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.647284 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.647285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.648318 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.648321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.648773 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.648774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.648888 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.649383 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.649391 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.650745 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.652274 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.652281 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.652283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.652539 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.652752 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.652753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.652753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.652846 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.653111 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.653117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.653232 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.653316 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.653468 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260807 15:07:05.654541 29086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:07:05.656162 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.648296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.18.129.60:44647)
0807 15:07:05.648455 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.648459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.648478 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:07:05.648496 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.648498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.648516 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.648589 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.649117 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.649862 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.650622 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.650992 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.651066 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.653379 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.653385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.653385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.653391 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.655011 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.655330 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.655332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.655333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.655386 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.655629 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.655629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.655630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.655752 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.655845 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.655953 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.656063 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.60@KRBTEST.COM for oryx/127.18.129.61@KRBTEST.COM
I20260807 15:07:05.661011 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.654873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.18.129.61:35837)
0807 15:07:05.654914 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.654915 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.654944 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:07:05.654970 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.654971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.654981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.655016 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.655273 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.655945 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.657046 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.657302 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.657349 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.658873 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.658875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.658875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.658883 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.660102 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.660418 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.660419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.660419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.660464 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.660622 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.660623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.660623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.660714 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.660757 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.660883 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.660947 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:07:05.661036 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.654784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.0.0.1:37690)
0807 15:07:05.654834 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.654864 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.654909 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.655031 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.655032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.655156 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.655200 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.655201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.656017 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.656018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.656963 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.656965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.657353 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.657354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.657403 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.657737 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.657742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.658769 (+ 1027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.660169 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.660170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.660171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.660359 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.660510 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.660511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.660511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.660570 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.660759 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.660759 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.660802 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.660867 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.660950 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:07:05.661387 29086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.60@KRBTEST.COM for oryx/127.18.129.60@KRBTEST.COM
I20260807 15:07:05.666685 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.661615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.0.0.1:59552)
0807 15:07:05.661648 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.661658 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.661710 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.661813 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.661814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.661948 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.661977 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.661978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.662501 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.662501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.663068 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.663070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.663381 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.663381 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.663413 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.663700 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.663703 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.664690 (+ 987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.665989 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.665990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.665990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.666128 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.666251 (+ 123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.666251 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.666251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.666292 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.666441 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.666441 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.666478 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.666546 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.666628 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 15:07:05.666760 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.661650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59552 (local address 127.18.129.60:44647)
0807 15:07:05.661681 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.661683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.661691 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:05.661763 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.661763 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.661771 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.661809 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.662028 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.662451 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.663131 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.663329 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.663378 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.664753 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.664754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.664754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.664760 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.665933 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.666170 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.666170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.666170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.666211 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.666327 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.666327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.666327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.666404 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.666444 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.666585 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.666671 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":13}
I20260807 15:07:05.668041 29086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Bootstrap starting.
I20260807 15:07:05.668891 29086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:05.669157 29086 log.cc:824] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:05.669834 29086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: No bootstrap required, opened a new log
I20260807 15:07:05.671314 29086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } }
I20260807 15:07:05.671435 29086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:05.671468 29086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8d99b921f5477889667008c53b93ed, State: Initialized, Role: FOLLOWER
I20260807 15:07:05.671587 29086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } }
I20260807 15:07:05.671859 29086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:05.671820 29092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } } }
I20260807 15:07:05.671916 29092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:07:05.673471 29112 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:07:05.673523 29112 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:07:05.675541 28988 master_runner.cc:428] Detected that this master 2a8d99b921f5477889667008c53b93ed is joining an existing cluster
I20260807 15:07:05.675603 28988 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.129.60:44647
I20260807 15:07:05.684259 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.677152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.0.0.1:59556)
0807 15:07:05.677479 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.677488 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.677559 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.677709 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.677710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.678019 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.678152 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.678155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.679095 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.679096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.680069 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.680072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.680541 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.680609 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.681422 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.681427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.681862 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.683350 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.683351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.683352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.683522 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.683695 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.683696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.683697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.683752 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.683942 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.683943 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.683981 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.684040 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.684146 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":34,"thread_start_us":88,"threads_started":1}
I20260807 15:07:05.684273 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.677313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.18.129.60:44647)
0807 15:07:05.677344 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.677346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.677538 (+ 192us) server_negotiation.cc:407] Connection header received
0807 15:07:05.677630 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.677630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.677638 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.677663 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.678235 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.679015 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680160 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.680429 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680467 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.681954 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.681955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.681955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.681961 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.683278 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.683593 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.683594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.683595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.683644 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.683799 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.683799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.683800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.683891 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.683943 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.684071 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.684199 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":16}
I20260807 15:07:05.702543 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.677323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37706 (local address 127.18.129.61:35837)
0807 15:07:05.677358 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.677360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.677493 (+ 133us) server_negotiation.cc:407] Connection header received
0807 15:07:05.677605 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.677606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.677616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.677645 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.678189 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.678977 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680160 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.680432 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680468 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.700083 (+ 19615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.700085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.700086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.700094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.701506 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.701816 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.701818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.701818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.701874 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.702039 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.702040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.702041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.702156 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.702210 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.702359 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.702443 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:07:05.702566 29119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.677145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.0.0.1:37706)
0807 15:07:05.677423 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.677434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.677521 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.677708 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.677709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.678034 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.678096 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.678098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.679074 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.679075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.680069 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.680072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.680553 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680553 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.680617 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.699416 (+ 18799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.699420 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.699979 (+ 559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.701586 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.701587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.701587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.701751 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.701928 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.701928 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.701929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.701982 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.702209 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.702210 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.702257 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.702324 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.702436 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":66,"thread_start_us":76,"threads_started":1}
I20260807 15:07:05.703070 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.677161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34554 (local address 127.18.129.62:46499)
0807 15:07:05.677209 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.677211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.677547 (+ 336us) server_negotiation.cc:407] Connection header received
0807 15:07:05.677636 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.677636 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.677647 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.677674 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.678248 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.679013 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680246 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.680520 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680556 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.700367 (+ 19811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.700369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.700370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.700377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.701868 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.702267 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.702268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.702269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.702330 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.702526 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.702527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.702527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.702658 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.702702 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.702911 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.702982 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:07:05.703086 29120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.677260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:34554)
0807 15:07:05.677493 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.677501 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.677569 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.677742 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.677742 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.678079 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.678167 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.678169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.679101 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.679102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.680171 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.680173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.680622 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.680623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.680675 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.699488 (+ 18813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.699492 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.699979 (+ 487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.701964 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.701965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.701965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.702173 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.702400 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.702401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.702401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.702457 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.702748 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.702749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.702791 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.702844 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.702969 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":91,"thread_start_us":163,"threads_started":1}
I20260807 15:07:05.707401 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 28088
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115224, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.62@KRBTEST.COM for oryx/127.18.129.60@KRBTEST.COM
I20260807 15:07:05.736790 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:36789
--local_ip_for_outbound_sockets=127.18.129.1
--tserver_master_addrs=127.18.129.62:46499
--webserver_port=42085
--webserver_interface=127.18.129.1
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab
--principal=oryx/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837,127.18.129.60:44647 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:07:05.739219 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.705050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59568 (local address 127.18.129.60:44647)
0807 15:07:05.705092 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.705095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.705103 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:05.705163 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.705164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.705173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.705230 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.705504 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.705945 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.706588 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.706803 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.706881 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.736416 (+ 29535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.736418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.736418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.736429 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.737913 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.738310 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.738311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.738312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.738377 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.738629 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.738629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.738630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.738779 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.738845 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.739042 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.739128 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:07:05.739233 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.705002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.0.0.1:59568)
0807 15:07:05.705049 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.705059 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.705097 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.705235 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.705236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.705398 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.705437 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.705439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.706010 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.706012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.706512 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.706514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.706903 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.706904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.706984 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.707371 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.707376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.736299 (+ 28923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.738002 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.738004 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.738004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.738215 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.738459 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.738461 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.738461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.738547 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.738873 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.738874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.738932 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.738999 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.739125 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:07:05.739799 28002 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.129.60:44647: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2a8d99b921f5477889667008c53b93ed"
member_type: NON_VOTER
last_known_addr {
host: "127.18.129.60"
port: 44647
}
attrs {
promote: true
}
}
dest_uuid: "0d381b52ca30418db8937a2848edd56a"
cas_config_opid_index: 5
I20260807 15:07:05.740057 28002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } }
I20260807 15:07:05.740398 28843 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2a8d99b921f5477889667008c53b93ed (127.18.129.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.741289 28843 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.129.60:44647
I20260807 15:07:05.741371 28843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.741443 28843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:05.741698 28988 master.cc:537] Master@127.18.129.60:44647 shutting down...
W20260807 15:07:05.743041 27979 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a -> Peer 2a8d99b921f5477889667008c53b93ed (127.18.129.60:44647): Couldn't send request to peer 2a8d99b921f5477889667008c53b93ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:07:05.745505 28988 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:05.745605 28988 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:05.745658 28988 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: stopping tablet replica
I20260807 15:07:05.747242 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.741714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.0.0.1:37720)
0807 15:07:05.742470 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.742481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.743779 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.744385 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.744387 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.744390 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:05.744436 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.744438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.745099 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.745100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.746368 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.746372 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.747001 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.747002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.747064 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.747086 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.747134 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.747175 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741}
I20260807 15:07:05.747540 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.741957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.18.129.61:35837)
0807 15:07:05.742435 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.742438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.743208 (+ 770us) server_negotiation.cc:407] Connection header received
0807 15:07:05.743873 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.743874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.743903 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.743955 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:05.744520 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.745023 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.746462 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.746925 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.746992 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.747072 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.747432 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.747477 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441}
I20260807 15:07:05.747746 28919 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 LEARNER]: Refusing update from remote peer 0d381b52ca30418db8937a2848edd56a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:07:05.747941 28847 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Connected to new peer: Peer: permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:07:05.748538 28919 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2a8d99b921f5477889667008c53b93ed (127.18.129.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.749469 28968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d381b52ca30418db8937a2848edd56a. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.749540 28968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: This master's current role is: LEARNER
I20260807 15:07:05.749560 28968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: NON_VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: true } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.749583 28968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: This master's current role is: LEARNER
I20260807 15:07:05.749779 28847 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: attempting to promote NON_VOTER 76d63311c9c9410c840cca74d203c16b to VOTER
I20260807 15:07:05.749989 28847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } }
I20260807 15:07:05.750617 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.750511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.0.0.1:59580)
0807 15:07:05.750546 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.750588 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 15:07:05.750686 28919 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 LEARNER]: Refusing update from remote peer 0d381b52ca30418db8937a2848edd56a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 15:07:05.750872 28843 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Connected to new peer: Peer: permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 15:07:05.751077 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.750950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.0.0.1:59594)
0807 15:07:05.751025 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.751052 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 15:07:05.751191 27979 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a -> Peer 2a8d99b921f5477889667008c53b93ed (127.18.129.60:44647): Couldn't send request to peer 2a8d99b921f5477889667008c53b93ed. Status: Network error: Client connection negotiation failed: client connection to 127.18.129.60:44647: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:07:05.751534 29126 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 76d63311c9c9410c840cca74d203c16b (127.18.129.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.751688 28919 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 76d63311c9c9410c840cca74d203c16b (127.18.129.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.752547 29126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.752575 28843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.752689 29126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:05.752703 28843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:05.752738 29125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.752905 29125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:07:05.761265 28988 master.cc:552] Master@127.18.129.60:44647 shutdown complete.
I20260807 15:07:05.761309 28988 master_runner.cc:305] Clearing existing system tablet
I20260807 15:07:05.761456 28988 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.761504 28988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.762084 28988 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.762526 29129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.762652 28988 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.762707 28988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.762732 28988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
uuid: "2a8d99b921f5477889667008c53b93ed"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "f7a1da6a76a3281f144d39022918eebd"
server_key_iv: "80b19cff0a511fd1f634d727f8e95851"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.763448 28988 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:05.764678 28988 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:07:05.764740 28988 log.cc:1196] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 15:07:05.764952 28988 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Deleting consensus metadata
I20260807 15:07:05.765064 28988 master_runner.cc:315] Copying system tablet from 127.18.129.62:46499
I20260807 15:07:05.765610 28988 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: tablet copy: Beginning tablet copy session from remote peer at address 127.18.129.62:46499
I20260807 15:07:05.771484 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.765930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34558 (local address 127.18.129.62:46499)
0807 15:07:05.765971 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.765973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.766038 (+ 65us) server_negotiation.cc:407] Connection header received
0807 15:07:05.766097 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.766098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.766109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.766160 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.766490 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.767139 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.767949 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.768192 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.768242 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.769150 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.769151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.769152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.769159 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.770449 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.770754 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.770755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.770755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.770811 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.770999 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.771001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.771001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.771108 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.771164 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.771297 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.771374 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:07:05.771499 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.765874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:34558)
0807 15:07:05.765988 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.765997 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.766051 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.766165 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.766166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.766364 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.766422 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.766424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.767206 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.767207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.767885 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.767888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.768261 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.768262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.768333 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.768766 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.768771 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.769069 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.770524 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.770525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.770525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.770692 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.770884 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.770884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.770885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.770947 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.771162 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.771163 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.771210 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.771268 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.771412 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260807 15:07:05.771947 28027 tablet_copy_service.cc:140] P 0d381b52ca30418db8937a2848edd56a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2a8d99b921f5477889667008c53b93ed ({username='oryx', principal='oryx/127.18.129.60@KRBTEST.COM'} at 127.0.0.1:34558)
I20260807 15:07:05.772023 28027 tablet_copy_service.cc:161] P 0d381b52ca30418db8937a2848edd56a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2a8d99b921f5477889667008c53b93ed at {username='oryx', principal='oryx/127.18.129.60@KRBTEST.COM'} at 127.0.0.1:34558: session id = 2a8d99b921f5477889667008c53b93ed-00000000000000000000000000000000
I20260807 15:07:05.772676 28027 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:07:05.773141 28988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:07:05.774201 28988 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: tablet copy: Starting download of 0 data blocks...
I20260807 15:07:05.774329 28988 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: tablet copy: Starting download of 1 WAL segments...
I20260807 15:07:05.775032 28988 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:07:05.775767 28027 tablet_copy_service.cc:342] P 0d381b52ca30418db8937a2848edd56a: Request end of tablet copy session 2a8d99b921f5477889667008c53b93ed-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.129.60@KRBTEST.COM'} at 127.0.0.1:34558
I20260807 15:07:05.775820 28027 tablet_copy_service.cc:433] P 0d381b52ca30418db8937a2848edd56a: ending tablet copy session 2a8d99b921f5477889667008c53b93ed-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2a8d99b921f5477889667008c53b93ed
I20260807 15:07:05.778169 28988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:05.779356 29138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:05.779562 29142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.779796 28988 server_base.cc:1034] running on GCE node
W20260807 15:07:05.782991 29139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.783138 28988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:05.783279 28988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:05.784392 28988 hybrid_clock.cc:648] HybridClock initialized: now 1786115225784402 us; error 41 us; skew 500 ppm
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:05.785729 28988 init.cc:377] Logged in from keytab as oryx/127.18.129.60@KRBTEST.COM (short username oryx)
I20260807 15:07:05.785993 28988 webserver.cc:460] Webserver started at http://127.18.129.60:39401/ using document root <none> and password file <none>
I20260807 15:07:05.786127 28988 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.786162 28988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.786751 28988 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.787292 29157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.787420 28988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.787462 28988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
uuid: "2a8d99b921f5477889667008c53b93ed"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "f7a1da6a76a3281f144d39022918eebd"
server_key_iv: "80b19cff0a511fd1f634d727f8e95851"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.787516 28988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.820992 28988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:05.822643 28988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:05.825646 28988 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.60:44647
I20260807 15:07:05.825670 29240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.60:44647 every 8 connection(s)
I20260807 15:07:05.825978 28988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
I20260807 15:07:05.826747 29241 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:07:05.827206 29241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Bootstrap starting.
I20260807 15:07:05.829406 29241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:07:05.829630 29241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: Bootstrap complete.
I20260807 15:07:05.829838 29241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } }
I20260807 15:07:05.829912 29241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2a8d99b921f5477889667008c53b93ed, State: Initialized, Role: LEARNER
I20260807 15:07:05.829949 29241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } }
I20260807 15:07:05.830123 29241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:05.830449 29247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:05.830628 29247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: This master's current role is: LEARNER
I20260807 15:07:05.831794 29260 catalog_manager.cc:1259] Loaded cluster ID: 37779a6cc96443ad963ac6ff683842dd
I20260807 15:07:05.831877 29260 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: loading cluster ID for follower catalog manager: success
I20260807 15:07:05.832949 29260 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: acquiring CA information for follower catalog manager: success
I20260807 15:07:05.833221 29260 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 15:07:05.853008 29122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:05.853226 29122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:05.853248 29122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:05.854343 29122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:05.854375 29122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:05.854399 29122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:05.855922 29122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab
--principal=oryx/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:36789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=42085
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837,127.18.129.60:44647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:05.856177 29122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:05.856406 29122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:05.861701 29267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:05.861685 29268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.861713 29122 server_base.cc:1034] running on GCE node
W20260807 15:07:05.861675 29270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:05.862116 29122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:05.862346 29122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:05.863484 29122 hybrid_clock.cc:648] HybridClock initialized: now 1786115225863469 us; error 27 us; skew 500 ppm
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:05.866026 29122 init.cc:377] Logged in from keytab as oryx/127.18.129.1@KRBTEST.COM (short username oryx)
I20260807 15:07:05.866484 29122 webserver.cc:460] Webserver started at http://127.18.129.1:42085/ using document root <none> and password file <none>
I20260807 15:07:05.866729 29122 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:05.866796 29122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:05.868285 29122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:05.868826 29282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.868995 29122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:05.869110 29122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "4ff6d3654b92431fb6c63e8f63b41a2d"
format_stamp: "Formatted at 2026-08-07 15:07:04 on dist-test-slave-5bn2"
server_key: "a2bcbe60daaff829d8b950b0d6502a66"
server_key_iv: "cc5fee8841d2e16123b7c6365fa2f0de"
server_key_version: "encryptionkey@0"
I20260807 15:07:05.869443 29122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:05.877364 29122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:05.879415 29122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:05.879554 29122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:05.879818 29122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:05.880132 29122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:05.880168 29122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.880198 29122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:05.880232 29122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:05.888862 29122 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:36789
I20260807 15:07:05.888898 29456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:36789 every 8 connection(s)
I20260807 15:07:05.889245 29122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:07:05.893291 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 29122
I20260807 15:07:05.893441 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 28285
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.61@KRBTEST.COM
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.60@KRBTEST.COM
I20260807 15:07:05.900635 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.889922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:46287 (local address 127.18.129.61:35837)
0807 15:07:05.889982 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.889985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.890294 (+ 309us) server_negotiation.cc:407] Connection header received
0807 15:07:05.890916 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.890918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.890931 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.890991 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.891876 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.892633 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.893780 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.894061 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.894099 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.897322 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.897323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.897323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.897330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.898773 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.899520 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.899522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.899522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.899596 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.899884 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.899886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.899886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.900037 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.900081 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.900448 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.900529 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:07:05.901154 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.889935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.18.129.1:46287)
0807 15:07:05.890173 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.890215 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.890776 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.891082 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.891086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.891653 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.891767 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.891773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.892723 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.892724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.893687 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.893690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.894187 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.894188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.894323 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.895012 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.895016 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.897216 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.898870 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.898879 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.898881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.899425 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.899672 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.899672 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.899672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.899808 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.900128 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.900135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.900296 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.900744 (+ 448us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.900917 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"spinlock_wait_cycles":7040,"thread_start_us":97,"threads_started":1}
I20260807 15:07:05.901552 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.889985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:47715 (local address 127.18.129.60:44647)
0807 15:07:05.890106 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.890108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.890554 (+ 446us) server_negotiation.cc:407] Connection header received
0807 15:07:05.891119 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.891121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.891135 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.891195 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.891871 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.892952 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.893861 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.894160 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.894213 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.897315 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.897317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.897317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.897326 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.899892 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.900236 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.900237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.900237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.900295 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.900489 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.900490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.900491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.900619 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.900895 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.901403 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.901475 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":56,"threads_started":1}
I20260807 15:07:05.901734 29471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.889937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.18.129.1:47715)
0807 15:07:05.890462 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.890472 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.890786 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.891245 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.891246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.891555 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.891767 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.891773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.893029 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.893030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.893789 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.893791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.894260 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.894260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.894323 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.894932 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.894942 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.897216 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.899966 (+ 2750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.899966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.899967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.900167 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.900371 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.900372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.900372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.900429 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.900926 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.900926 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.900966 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.901030 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.901127 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":116,"threads_started":1}
I20260807 15:07:05.902230 29457 heartbeater.cc:347] Connected to a master server at 127.18.129.60:44647
I20260807 15:07:05.902326 29457 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:05.902503 29457 heartbeater.cc:511] Master 127.18.129.60:44647 requested a full tablet report, sending...
I20260807 15:07:05.903038 29187 ts_manager.cc:194] Registered new tserver with Master: 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789)
Aug 07 15:07:05 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115225, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:05.909852 29463 heartbeater.cc:347] Connected to a master server at 127.18.129.61:35837
I20260807 15:07:05.909909 29463 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:05.909974 29463 heartbeater.cc:511] Master 127.18.129.61:35837 requested a full tablet report, sending...
I20260807 15:07:05.910610 28902 ts_manager.cc:194] Registered new tserver with Master: 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789)
I20260807 15:07:05.912441 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.890332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:49087 (local address 127.18.129.62:46499)
0807 15:07:05.890390 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:05.890392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:05.891035 (+ 643us) server_negotiation.cc:407] Connection header received
0807 15:07:05.891078 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:05.891079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:05.891091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:05.891138 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:05.891853 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.892494 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.907461 (+ 14967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.907714 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.907749 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.910025 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:05.910027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:05.910027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:05.910037 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:05.911435 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.911739 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.911740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.911741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.911798 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:05.911960 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:05.911962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:05.911962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:05.912083 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:05.912141 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:05.912273 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 15:07:05.912352 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:07:05.912479 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:05.890258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.1:49087)
0807 15:07:05.890978 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:05.890985 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:05.891028 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:05.891141 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:05.891142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:05.891559 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:05.891766 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.891771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.892607 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.892608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.907355 (+ 14747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:05.907359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:05.907818 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:05.907818 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:05.907875 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:05.908297 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:05.908303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:05.909911 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:05.911510 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.911511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.911511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.911677 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.911852 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:05.911853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:05.911853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:05.911911 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:05.912140 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:05.912140 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:05.912185 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:07:05.912246 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:07:05.912353 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":165,"threads_started":1}
I20260807 15:07:05.912851 29462 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46499
I20260807 15:07:05.912914 29462 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:05.912981 29462 heartbeater.cc:511] Master 127.18.129.62:46499 requested a full tablet report, sending...
I20260807 15:07:05.913362 28002 ts_manager.cc:194] Re-registered known tserver with Master: 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789)
I20260807 15:07:05.913754 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:43269
--local_ip_for_outbound_sockets=127.18.129.2
--tserver_master_addrs=127.18.129.62:46499
--webserver_port=38405
--webserver_interface=127.18.129.2
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab
--principal=oryx/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837,127.18.129.60:44647 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:07:05.913791 28002 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:49087
W20260807 15:07:05.995687 29472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:05.995877 29472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:05.995899 29472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:05.997006 29472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:05.997047 29472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:05.997076 29472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:05.998531 29472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab
--principal=oryx/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:43269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=38405
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837,127.18.129.60:44647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:05.998744 29472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:05.998971 29472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:06.003937 29480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:06.003949 29478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:06.003981 29472 server_base.cc:1034] running on GCE node
W20260807 15:07:06.003985 29482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:06.004305 29472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:06.004544 29472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:06.005683 29472 hybrid_clock.cc:648] HybridClock initialized: now 1786115226005670 us; error 30 us; skew 500 ppm
Aug 07 15:07:06 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115226, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:06.008149 29472 init.cc:377] Logged in from keytab as oryx/127.18.129.2@KRBTEST.COM (short username oryx)
I20260807 15:07:06.008569 29472 webserver.cc:460] Webserver started at http://127.18.129.2:38405/ using document root <none> and password file <none>
I20260807 15:07:06.008765 29472 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:06.008833 29472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:06.010118 29472 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:06.010640 29497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:06.010794 29472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:06.010901 29472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "1ca0652f5f53478ea3dd9ac1fd968025"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "135bb3dbfffd748c119cae316140e6af"
server_key_iv: "d2e4e18e644a3d9fd2bb401b2645296f"
server_key_version: "encryptionkey@0"
I20260807 15:07:06.011245 29472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:06.024562 29472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:06.026190 29472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:06.026330 29472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:06.026547 29472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:06.026840 29472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:06.026906 29472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:06.026937 29472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:06.026960 29472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:06.034065 29472 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43269
I20260807 15:07:06.034147 29687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43269 every 8 connection(s)
I20260807 15:07:06.034448 29472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:07:06.039079 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 29472
I20260807 15:07:06.039230 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 28492
Aug 07 15:07:06 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115226, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.61@KRBTEST.COM
Aug 07 15:07:06 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115226, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.60@KRBTEST.COM
Aug 07 15:07:06 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115226, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:06.046376 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.035302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:46025 (local address 127.18.129.62:46499)
0807 15:07:06.035348 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.035350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.035717 (+ 367us) server_negotiation.cc:407] Connection header received
0807 15:07:06.036734 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.036735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.036747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.036776 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.037508 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.038275 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039258 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.039540 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039591 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.043049 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.043050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.043051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.043058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.044658 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045235 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.045236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.045237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.045296 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045610 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.045611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.045611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.045749 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.045792 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.046188 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.046275 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:07:06.046376 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.035602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:38089 (local address 127.18.129.61:35837)
0807 15:07:06.035643 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.035645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.035750 (+ 105us) server_negotiation.cc:407] Connection header received
0807 15:07:06.036730 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.036731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.036745 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.036776 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.037495 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.037981 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039249 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.039547 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039587 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.043040 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.043041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.043041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.043049 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.044823 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045234 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.045235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.045236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.045296 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045609 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.045611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.045611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.045748 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.045792 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.046188 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.046275 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:07:06.047020 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.035521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.18.129.2:38089)
0807 15:07:06.035694 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.035703 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.036617 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.036850 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.036854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.037276 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:06.037389 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.037396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.038083 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.038084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.039150 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.039152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.039730 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.039843 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.040379 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:06.040387 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:06.042932 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:06.044897 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.044897 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.044898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.045158 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.045369 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045369 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.045370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.045528 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.045839 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:06.045847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:06.046014 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.046512 (+ 498us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.046784 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":191,"thread_start_us":69,"threads_started":1}
I20260807 15:07:06.047029 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.035165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.2:46025)
0807 15:07:06.035567 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.035611 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.036610 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.036850 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.036854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.037274 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:06.037389 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.037396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.038378 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.038379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.039148 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.039151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.039684 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.039846 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.040757 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:06.040763 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:06.042933 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:06.044753 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.044761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.044763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.045144 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.045369 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.045370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.045528 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.045839 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:06.045847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:06.046014 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.046513 (+ 499us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.046721 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":29,"thread_start_us":97,"threads_started":1}
I20260807 15:07:06.047065 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.035157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:37975 (local address 127.18.129.60:44647)
0807 15:07:06.035219 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.035222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.035698 (+ 476us) server_negotiation.cc:407] Connection header received
0807 15:07:06.036706 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.036707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.036721 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.036768 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.037485 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.038262 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039287 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.039588 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039624 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.042541 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.042543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.042543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.042553 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.045321 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045805 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.045807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.045807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.045879 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.046128 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.046129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.046129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.046289 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.046340 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.046812 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.046955 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:07:06.047474 29690 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46499
I20260807 15:07:06.047555 29690 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:06.047695 29690 heartbeater.cc:511] Master 127.18.129.62:46499 requested a full tablet report, sending...
I20260807 15:07:06.047927 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.035164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.18.129.2:37975)
0807 15:07:06.035583 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.035612 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.036618 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.036849 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.036853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.037203 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:06.037389 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.037393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.038349 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.038350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.039201 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.039203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.039693 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.039693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.039843 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.040631 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:06.040636 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:06.042438 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:06.045428 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.045430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.045701 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.045968 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.045968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.045969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.046040 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.046398 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:06.046399 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:06.046664 (+ 265us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.047645 (+ 981us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.047800 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":23,"thread_start_us":93,"threads_started":1}
I20260807 15:07:06.048074 28002 ts_manager.cc:194] Re-registered known tserver with Master: 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269)
I20260807 15:07:06.048422 28002 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:46025
I20260807 15:07:06.063369 29691 heartbeater.cc:347] Connected to a master server at 127.18.129.61:35837
I20260807 15:07:06.063472 29691 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:06.063598 29691 heartbeater.cc:511] Master 127.18.129.61:35837 requested a full tablet report, sending...
I20260807 15:07:06.063812 29688 heartbeater.cc:347] Connected to a master server at 127.18.129.60:44647
I20260807 15:07:06.063872 29688 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:06.063926 29688 heartbeater.cc:511] Master 127.18.129.60:44647 requested a full tablet report, sending...
I20260807 15:07:06.063977 28902 ts_manager.cc:194] Registered new tserver with Master: 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269)
I20260807 15:07:06.064312 29187 ts_manager.cc:194] Registered new tserver with Master: 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269)
I20260807 15:07:06.067123 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:46147
--local_ip_for_outbound_sockets=127.18.129.3
--tserver_master_addrs=127.18.129.62:46499
--webserver_port=43671
--webserver_interface=127.18.129.3
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab
--principal=oryx/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837,127.18.129.60:44647 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:06.148841 29702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:06.149048 29702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:06.149087 29702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:06.150272 29702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:06.150316 29702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:06.150349 29702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:06.151847 29702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab
--principal=oryx/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:46147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=43671
--tserver_master_addrs=127.18.129.62:46499,127.18.129.61:35837,127.18.129.60:44647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:06.152087 29702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:06.152194 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.149558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.0.0.1:59600)
0807 15:07:06.149612 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.149623 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.149666 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.149849 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.149851 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.149853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:06.149908 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.149910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.150603 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.150605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.151349 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.151352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.151938 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.151939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.152004 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.152008 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.152063 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.152110 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 15:07:06.152194 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.149648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.18.129.60:44647)
0807 15:07:06.149723 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.149725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.149733 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:06.149758 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.149759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.149779 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.149832 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:06.149983 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.150512 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.151443 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.151842 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.151906 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.151976 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.152088 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.152126 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:07:06.152333 29702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:07:06.152572 29198 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 15:07:06.152705 29247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d381b52ca30418db8937a2848edd56a. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:06.152807 29247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: This master's current role is: LEARNER
W20260807 15:07:06.157459 29708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:06.157465 29711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:06.157478 29707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:06.157665 29702 server_base.cc:1034] running on GCE node
I20260807 15:07:06.157931 29702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:06.158165 29702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:06.159312 29702 hybrid_clock.cc:648] HybridClock initialized: now 1786115226159298 us; error 26 us; skew 500 ppm
Aug 07 15:07:06 dist-test-slave-5bn2 krb5kdc[27929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115226, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:06.161820 29702 init.cc:377] Logged in from keytab as oryx/127.18.129.3@KRBTEST.COM (short username oryx)
I20260807 15:07:06.162262 29702 webserver.cc:460] Webserver started at http://127.18.129.3:43671/ using document root <none> and password file <none>
I20260807 15:07:06.162457 29702 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:06.162521 29702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:06.163856 29702 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:06.164403 29718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:06.164553 29702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:06.164640 29702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "7d97875043184639b13b66f88a153466"
format_stamp: "Formatted at 2026-08-07 15:07:05 on dist-test-slave-5bn2"
server_key: "cf09ad468b418b317e4a651e179c2c03"
server_key_iv: "95ccf32c2dff00c3fcd8aa84045d1573"
server_key_version: "encryptionkey@0"
I20260807 15:07:06.164968 29702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:06.171407 29702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:06.173360 29702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:06.173483 29702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:06.173678 29702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:06.173944 29702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:06.173977 29702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:06.173997 29702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:06.174021 29702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:06.179601 29702 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:46147
I20260807 15:07:06.179661 29892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:46147 every 8 connection(s)
I20260807 15:07:06.179940 29702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:07:06.182487 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 29702
Aug 07 15:07:06 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115226, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.61@KRBTEST.COM
Aug 07 15:07:06 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115226, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.60@KRBTEST.COM
Aug 07 15:07:06 dist-test-slave-5bn2 krb5kdc[27929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115226, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:06.191450 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.180668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.18.129.3:55631)
0807 15:07:06.180900 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.180909 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.181464 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.181703 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.181703 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.182147 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:06.182417 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.182423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.183443 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.183444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.184723 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.184726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.185244 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.185244 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.185311 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.186007 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:06.186012 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:06.188066 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:06.189811 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.189818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.189820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.190211 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.190393 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.190394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.190394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.190513 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.190764 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:06.190770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:06.190945 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.191048 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.191227 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":100,"spinlock_wait_cycles":7936,"thread_start_us":97,"threads_started":1}
I20260807 15:07:06.191742 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.180647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:55631 (local address 127.18.129.60:44647)
0807 15:07:06.180689 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.180691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.180970 (+ 279us) server_negotiation.cc:407] Connection header received
0807 15:07:06.181593 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.181595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.181607 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.181638 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.182529 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.183317 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.184828 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.185126 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.185165 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.188173 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.188175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.188175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.188182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.189688 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.190271 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.190272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.190273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.190333 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.190573 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.190574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.190574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.190686 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.190739 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.191578 (+ 839us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.191658 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:07:06.192212 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.181245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:37509 (local address 127.18.129.61:35837)
0807 15:07:06.181293 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.181295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.181369 (+ 74us) server_negotiation.cc:407] Connection header received
0807 15:07:06.181549 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.181550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.181562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.181611 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.182529 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.183279 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.184825 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.185019 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.185058 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.188193 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.188194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.188194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.188201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.189796 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.190920 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.190921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.190921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.190985 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.191185 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.191186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.191187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.191309 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.191352 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.192045 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.192122 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:07:06.192225 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.181172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.18.129.3:37509)
0807 15:07:06.181305 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.181315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.181473 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.181678 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.181682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.182236 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:06.182417 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.182424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.183413 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.183413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.184727 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.184730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.185129 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.185130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.185227 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.185676 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:06.185686 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:06.188101 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:06.189882 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.189883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.189884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.190791 (+ 907us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.191058 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.191059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.191059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.191118 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.191395 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:06.191396 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:06.191433 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.192010 (+ 577us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.192118 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":74,"thread_start_us":80,"threads_started":1}
I20260807 15:07:06.192394 29894 heartbeater.cc:347] Connected to a master server at 127.18.129.60:44647
I20260807 15:07:06.193271 29894 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:06.193468 29894 heartbeater.cc:511] Master 127.18.129.60:44647 requested a full tablet report, sending...
I20260807 15:07:06.193924 29187 ts_manager.cc:194] Registered new tserver with Master: 7d97875043184639b13b66f88a153466 (127.18.129.3:46147)
I20260807 15:07:06.193965 29897 heartbeater.cc:347] Connected to a master server at 127.18.129.61:35837
I20260807 15:07:06.194007 29897 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:06.194067 29897 heartbeater.cc:511] Master 127.18.129.61:35837 requested a full tablet report, sending...
I20260807 15:07:06.194347 28902 ts_manager.cc:194] Registered new tserver with Master: 7d97875043184639b13b66f88a153466 (127.18.129.3:46147)
I20260807 15:07:06.194358 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.183506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.18.129.62:46499)
0807 15:07:06.184436 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.184438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.185565 (+ 1127us) server_negotiation.cc:407] Connection header received
0807 15:07:06.185629 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.185630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.185641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.185701 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.186417 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.187164 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.188299 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.188569 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.188633 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.191370 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.191371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.191372 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.191379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.193009 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.193374 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.193375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.193376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.193439 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.193644 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.193645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.193645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.193772 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.193857 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.194186 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.194263 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":101,"threads_started":1}
I20260807 15:07:06.195930 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.180917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:46415 (local address 127.18.129.62:46499)
0807 15:07:06.180970 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.180972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.180983 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:07:06.181593 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.181594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.181607 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.181638 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.182529 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.183319 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.184828 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.185131 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.185165 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.190222 (+ 5057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.190223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.190224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.190231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.191904 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.195187 (+ 3283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.195188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.195188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.195247 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.195449 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.195449 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.195450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.195552 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.195610 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.195774 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.195843 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:07:06.196702 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.180668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.3:46415)
0807 15:07:06.180839 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.180887 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.181468 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.181727 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.181728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.182155 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:06.182417 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.182423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.183415 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.183415 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.184723 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.184726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.185242 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.185243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.185320 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.187889 (+ 2569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:06.187894 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:06.190022 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:06.194894 (+ 4872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.194895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.194896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.195096 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.195318 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:06.195318 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:06.195319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:06.195370 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:06.195614 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:06.195615 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:06.195656 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.196321 (+ 665us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.196424 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":107,"thread_start_us":166,"threads_started":1}
I20260807 15:07:06.197192 29895 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46499
I20260807 15:07:06.197261 29895 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:06.197329 29895 heartbeater.cc:511] Master 127.18.129.62:46499 requested a full tablet report, sending...
I20260807 15:07:06.197541 28003 ts_manager.cc:194] Re-registered known tserver with Master: 7d97875043184639b13b66f88a153466 (127.18.129.3:46147)
I20260807 15:07:06.197810 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.184187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.18.129.61:35837)
0807 15:07:06.185670 (+ 1483us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.185671 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.185843 (+ 172us) server_negotiation.cc:407] Connection header received
0807 15:07:06.185863 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.185864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.185875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.185932 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.186551 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.187340 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.188345 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.189393 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.189455 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.192989 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.192990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.192991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.192997 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.196316 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.196721 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.196722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.196723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.196783 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.196954 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.196954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.196955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.197203 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.197280 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.197643 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.197725 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1464,"thread_start_us":58,"threads_started":1}
I20260807 15:07:06.197957 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.184615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59616 (local address 127.18.129.60:44647)
0807 15:07:06.186563 (+ 1948us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.186565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.186679 (+ 114us) server_negotiation.cc:407] Connection header received
0807 15:07:06.186761 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.186762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.186774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.186808 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.187562 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.190638 (+ 3076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.192890 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.193193 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.193250 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.194563 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.194565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.194565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.194572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.196271 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.196682 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.196684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.196684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.196747 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.196933 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.196934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.196935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.197057 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.197153 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.197773 (+ 620us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.197850 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1929,"thread_start_us":59,"threads_started":1}
I20260807 15:07:06.198012 28003 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:46415
I20260807 15:07:06.196229 28002 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:07:06.198486 28002 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:06.199241 29126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: NON_VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: true } } }
I20260807 15:07:06.199337 29126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:06.201407 29917 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: attempting to promote NON_VOTER 2a8d99b921f5477889667008c53b93ed to VOTER
I20260807 15:07:06.201537 29917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } }
I20260807 15:07:06.201982 28919 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Refusing update from remote peer 0d381b52ca30418db8937a2848edd56a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260807 15:07:06.201993 29197 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 LEARNER]: Refusing update from remote peer 0d381b52ca30418db8937a2848edd56a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260807 15:07:06.202236 29917 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Connected to new peer: Peer: permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260807 15:07:06.202330 28847 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 15:07:06.203490 28847 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 2a8d99b921f5477889667008c53b93ed (127.18.129.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:06.203650 29197 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 2a8d99b921f5477889667008c53b93ed (127.18.129.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:06.204392 28847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:06.204476 28847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:06.204473 29247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:06.204524 28847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:06.204550 29247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:07:06.204577 28847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:06.204607 28847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:06.204639 28847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:06.204872 28919 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 2a8d99b921f5477889667008c53b93ed (127.18.129.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:06.205715 29125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d381b52ca30418db8937a2848edd56a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:06.205803 29125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:07:06.206776 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.202536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43269 (local address 127.0.0.1:54552)
0807 15:07:06.202753 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.202763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.202796 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.203193 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.203194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.203196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:06.203236 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.203238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.204529 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.204530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.205571 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.205575 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.206562 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.206563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.206623 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.206627 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.206677 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.206713 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260807 15:07:06.206811 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.202662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.18.129.2:43269)
0807 15:07:06.203049 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.203054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.203064 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:07:06.203097 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.203099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.203111 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.203183 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:06.203312 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.204393 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.205728 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.206485 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.206551 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.206632 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.206688 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.206724 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":75,"threads_started":1}
I20260807 15:07:06.207489 29587 tablet_service.cc:1467] Processing CreateTablet for tablet 967378f9d849473bb1fc5b93e5a40189 (DEFAULT_TABLE table=test-table [id=9ac8c119f7a44547b33c8e82b1a5b65f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:06.207727 29938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.202963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.18.129.1:36789)
0807 15:07:06.203127 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.203131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.203560 (+ 429us) server_negotiation.cc:407] Connection header received
0807 15:07:06.204746 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.204750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.204769 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.204841 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:06.205011 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.205670 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.206701 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.207409 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.207464 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.207516 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.207630 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.207657 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260807 15:07:06.207732 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.202622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36789 (local address 127.0.0.1:51544)
0807 15:07:06.203441 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.203451 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.203499 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.204912 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.204913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.204915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:06.204956 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.204957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.205762 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.205763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.206601 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.206603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.207487 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.207488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.207552 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.207557 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.207620 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.207661 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":66,"threads_started":1}
I20260807 15:07:06.207894 29587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967378f9d849473bb1fc5b93e5a40189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:06.210548 29939 tablet_bootstrap.cc:492] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Bootstrap starting.
I20260807 15:07:06.211246 29939 tablet_bootstrap.cc:654] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:06.211422 29343 tablet_service.cc:1467] Processing CreateTablet for tablet 967378f9d849473bb1fc5b93e5a40189 (DEFAULT_TABLE table=test-table [id=9ac8c119f7a44547b33c8e82b1a5b65f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:06.211565 29939 log.cc:824] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:06.211761 29343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967378f9d849473bb1fc5b93e5a40189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:06.212371 29939 tablet_bootstrap.cc:492] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: No bootstrap required, opened a new log
I20260807 15:07:06.212428 29939 ts_tablet_manager.cc:1397] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:06.212978 29931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.202562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:46147 (local address 127.0.0.1:33720)
0807 15:07:06.203352 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.203363 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.206198 (+ 2835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.206420 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.206421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.206422 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:06.206462 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.206464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.211291 (+ 4827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.211292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.212119 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.212122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.212744 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.212745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.212807 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.212810 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.212863 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.212902 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":123,"threads_started":1}
I20260807 15:07:06.212981 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.202827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.18.129.3:46147)
0807 15:07:06.203297 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.203301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.206166 (+ 2865us) server_negotiation.cc:407] Connection header received
0807 15:07:06.206290 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.206292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.206306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.206369 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:06.206544 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.211217 (+ 4673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.212196 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.212667 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.212730 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.212797 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.212877 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.212913 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":90,"threads_started":1}
I20260807 15:07:06.213616 29797 tablet_service.cc:1467] Processing CreateTablet for tablet 967378f9d849473bb1fc5b93e5a40189 (DEFAULT_TABLE table=test-table [id=9ac8c119f7a44547b33c8e82b1a5b65f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:06.213904 29797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967378f9d849473bb1fc5b93e5a40189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:06.214311 29939 raft_consensus.cc:348] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.214462 29939 raft_consensus.cc:374] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:06.214494 29939 raft_consensus.cc:729] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca0652f5f53478ea3dd9ac1fd968025, State: Initialized, Role: FOLLOWER
I20260807 15:07:06.214619 29939 consensus_queue.cc:260] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.215021 29939 ts_tablet_manager.cc:1428] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:07:06.215703 29944 tablet_bootstrap.cc:492] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Bootstrap starting.
I20260807 15:07:06.216317 29944 tablet_bootstrap.cc:654] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:06.216590 29944 log.cc:824] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:06.216779 29945 tablet_bootstrap.cc:492] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Bootstrap starting.
I20260807 15:07:06.216815 29690 heartbeater.cc:503] Master 127.18.129.62:46499 was elected leader, sending a full tablet report...
I20260807 15:07:06.217281 29944 tablet_bootstrap.cc:492] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: No bootstrap required, opened a new log
I20260807 15:07:06.217348 29944 ts_tablet_manager.cc:1397] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:06.217685 29945 tablet_bootstrap.cc:654] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:06.217949 29945 log.cc:824] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:06.218554 29945 tablet_bootstrap.cc:492] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: No bootstrap required, opened a new log
I20260807 15:07:06.218612 29945 ts_tablet_manager.cc:1397] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:06.218825 29944 raft_consensus.cc:348] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.219033 29944 raft_consensus.cc:374] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:06.219059 29944 raft_consensus.cc:729] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff6d3654b92431fb6c63e8f63b41a2d, State: Initialized, Role: FOLLOWER
I20260807 15:07:06.219164 29944 consensus_queue.cc:260] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.219390 29944 ts_tablet_manager.cc:1428] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:07:06.219447 29462 heartbeater.cc:503] Master 127.18.129.62:46499 was elected leader, sending a full tablet report...
I20260807 15:07:06.221175 29945 raft_consensus.cc:348] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.221294 29945 raft_consensus.cc:374] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:06.221323 29945 raft_consensus.cc:729] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d97875043184639b13b66f88a153466, State: Initialized, Role: FOLLOWER
I20260807 15:07:06.221428 29945 consensus_queue.cc:260] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.221637 29945 ts_tablet_manager.cc:1428] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:07:06.221709 29895 heartbeater.cc:503] Master 127.18.129.62:46499 was elected leader, sending a full tablet report...
W20260807 15:07:06.285523 29695 tablet.cc:2369] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:07:06.390143 29464 tablet.cc:2369] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:07:06.391230 29949 raft_consensus.cc:482] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:06.391372 29949 raft_consensus.cc:504] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.391764 29949 leader_election.cc:288] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269), 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789)
I20260807 15:07:06.395484 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.391936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:60327 (local address 127.18.129.2:43269)
0807 15:07:06.391981 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.391983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.391996 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:07:06.392111 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.392114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.392125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.392173 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:06.392372 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.393122 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.394560 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.395153 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.395212 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.395218 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.395375 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.395411 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:07:06.395484 29938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.392090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:60835 (local address 127.18.129.1:36789)
0807 15:07:06.392140 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.392142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.392150 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:06.392176 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.392177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.392185 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.392228 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:06.392376 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.393117 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.394563 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.395133 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.395208 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.395218 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.395375 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.395411 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:07:06.395514 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.392020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36789 (local address 127.18.129.3:60835)
0807 15:07:06.392054 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.392064 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.392112 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.392237 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.392237 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.392239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:06.392283 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.392285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.393197 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.393197 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.394477 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.394480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.395206 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.395208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.395303 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.395308 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.395364 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.395408 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":51}
I20260807 15:07:06.395516 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.391908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43269 (local address 127.18.129.3:60327)
0807 15:07:06.391958 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:06.391970 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:06.392014 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:06.392199 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:06.392200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:06.392202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:06.392284 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.392285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.393196 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.393197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.394477 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:06.394480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.395215 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.395216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:06.395309 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.395311 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:07:06.395365 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:07:06.395408 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":57}
I20260807 15:07:06.395984 29384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967378f9d849473bb1fc5b93e5a40189" candidate_uuid: "7d97875043184639b13b66f88a153466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" is_pre_election: true
I20260807 15:07:06.396006 29612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967378f9d849473bb1fc5b93e5a40189" candidate_uuid: "7d97875043184639b13b66f88a153466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" is_pre_election: true
I20260807 15:07:06.396114 29384 raft_consensus.cc:2454] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d97875043184639b13b66f88a153466 in term 0.
I20260807 15:07:06.396137 29612 raft_consensus.cc:2454] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d97875043184639b13b66f88a153466 in term 0.
I20260807 15:07:06.396337 29721 leader_election.cc:302] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca0652f5f53478ea3dd9ac1fd968025, 7d97875043184639b13b66f88a153466; no voters:
I20260807 15:07:06.396458 29949 raft_consensus.cc:2781] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:06.396515 29949 raft_consensus.cc:482] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:06.396538 29949 raft_consensus.cc:3037] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:06.397233 29949 raft_consensus.cc:504] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.397344 29949 leader_election.cc:288] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [CANDIDATE]: Term 1 election: Requested vote from peers 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269), 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789)
I20260807 15:07:06.397488 29612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967378f9d849473bb1fc5b93e5a40189" candidate_uuid: "7d97875043184639b13b66f88a153466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca0652f5f53478ea3dd9ac1fd968025"
I20260807 15:07:06.397535 29384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "967378f9d849473bb1fc5b93e5a40189" candidate_uuid: "7d97875043184639b13b66f88a153466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d"
I20260807 15:07:06.397586 29612 raft_consensus.cc:3037] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:06.397598 29384 raft_consensus.cc:3037] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:06.398471 29384 raft_consensus.cc:2454] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d97875043184639b13b66f88a153466 in term 1.
I20260807 15:07:06.398471 29612 raft_consensus.cc:2454] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d97875043184639b13b66f88a153466 in term 1.
I20260807 15:07:06.398649 29721 leader_election.cc:302] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca0652f5f53478ea3dd9ac1fd968025, 7d97875043184639b13b66f88a153466; no voters:
I20260807 15:07:06.398746 29949 raft_consensus.cc:2781] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:06.398947 29949 raft_consensus.cc:686] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 1 LEADER]: Becoming Leader. State: Replica: 7d97875043184639b13b66f88a153466, State: Running, Role: LEADER
I20260807 15:07:06.399036 29949 consensus_queue.cc:237] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:06.399907 28002 catalog_manager.cc:5495] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d97875043184639b13b66f88a153466 (127.18.129.3). New cstate: current_term: 1 leader_uuid: "7d97875043184639b13b66f88a153466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } health_report { overall_health: HEALTHY } } }
I20260807 15:07:06.429621 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.427348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.18.129.3:46147)
0807 15:07:06.427401 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.427403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.427411 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:06.427455 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.427455 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.427477 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.427547 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:06.427683 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.428154 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.428872 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.429078 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.429136 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.429299 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:06.429402 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:06.429474 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.429510 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.429541 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 15:07:06.430663 29899 tablet.cc:2369] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:07:06.431547 29384 raft_consensus.cc:1261] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 1 FOLLOWER]: Refusing update from remote peer 7d97875043184639b13b66f88a153466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:06.431546 29612 raft_consensus.cc:1261] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 1 FOLLOWER]: Refusing update from remote peer 7d97875043184639b13b66f88a153466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:06.431874 29950 consensus_queue.cc:1035] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:06.432008 29949 consensus_queue.cc:1035] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:06.435563 29959 mvcc.cc:205] Tried to move back new op lower bound from 7315927967460503552 to 7315927967331295232. Current Snapshot: MvccSnapshot[applied={T|T < 7315927967460503552 or (T in {7315927967460503552})}]
I20260807 15:07:06.435675 29960 mvcc.cc:205] Tried to move back new op lower bound from 7315927967460503552 to 7315927967331295232. Current Snapshot: MvccSnapshot[applied={T|T < 7315927967460503552}]
I20260807 15:07:06.437989 28003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:06.438072 28003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:06.439515 28003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d381b52ca30418db8937a2848edd56a: Sending DeleteTablet for 3 replicas of tablet 967378f9d849473bb1fc5b93e5a40189
I20260807 15:07:06.439901 29343 tablet_service.cc:1514] Processing DeleteTablet for tablet 967378f9d849473bb1fc5b93e5a40189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:06 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:51544
I20260807 15:07:06.439990 29587 tablet_service.cc:1514] Processing DeleteTablet for tablet 967378f9d849473bb1fc5b93e5a40189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:06 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:54552
I20260807 15:07:06.439997 29797 tablet_service.cc:1514] Processing DeleteTablet for tablet 967378f9d849473bb1fc5b93e5a40189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:06 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:33720
I20260807 15:07:06.440261 29972 tablet_replica.cc:330] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: stopping tablet replica
I20260807 15:07:06.440310 29973 tablet_replica.cc:330] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: stopping tablet replica
I20260807 15:07:06.440415 29973 raft_consensus.cc:2229] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:07:06.440443 29972 raft_consensus.cc:2229] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:06.440543 29973 raft_consensus.cc:2258] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:06.440676 29972 raft_consensus.cc:2258] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:06.440960 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 27950
I20260807 15:07:06.441078 29973 ts_tablet_manager.cc:1905] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:06.442718 29973 ts_tablet_manager.cc:1918] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:06.442782 29973 log.cc:1196] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/967378f9d849473bb1fc5b93e5a40189
I20260807 15:07:06.443003 29973 ts_tablet_manager.cc:1939] T 967378f9d849473bb1fc5b93e5a40189 P 7d97875043184639b13b66f88a153466: Deleting consensus metadata
I20260807 15:07:06.449460 29971 tablet_replica.cc:330] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: stopping tablet replica
I20260807 15:07:06.449604 29971 raft_consensus.cc:2229] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:06.449728 29971 raft_consensus.cc:2258] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:06.450326 29971 ts_tablet_manager.cc:1905] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:06.452003 29971 ts_tablet_manager.cc:1918] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:06.452085 29971 log.cc:1196] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/967378f9d849473bb1fc5b93e5a40189
I20260807 15:07:06.452340 29971 ts_tablet_manager.cc:1939] T 967378f9d849473bb1fc5b93e5a40189 P 4ff6d3654b92431fb6c63e8f63b41a2d: Deleting consensus metadata
W20260807 15:07:06.455355 29283 connection.cc:535] server connection from 127.0.0.1:51544 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 15:07:06.455392 29722 connection.cc:535] server connection from 127.0.0.1:33720 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:07:06.456784 29972 ts_tablet_manager.cc:1905] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:06.458511 29972 ts_tablet_manager.cc:1918] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:06.458606 29972 log.cc:1196] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/967378f9d849473bb1fc5b93e5a40189
I20260807 15:07:06.458906 29972 ts_tablet_manager.cc:1939] T 967378f9d849473bb1fc5b93e5a40189 P 1ca0652f5f53478ea3dd9ac1fd968025: Deleting consensus metadata
W20260807 15:07:06.459244 29504 connection.cc:441] server connection from 127.0.0.1:54552 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54552 (request call id 1) could send its response
I20260807 15:07:06.461004 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.440667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59620 (local address 127.18.129.60:44647)
0807 15:07:06.440719 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.440721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.441009 (+ 288us) server_negotiation.cc:407] Connection header received
0807 15:07:06.449388 (+ 8379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.449389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.449410 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.449497 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.453305 (+ 3808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.454115 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.455390 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.455729 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.455773 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.457743 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.457745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.457745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.457753 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.459741 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.460186 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.460187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.460188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.460246 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.460428 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.460429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.460429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.460551 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.460631 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.460814 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.460916 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:07:06.465003 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.457499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59624 (local address 127.18.129.60:44647)
0807 15:07:06.457543 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.457545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.457865 (+ 320us) server_negotiation.cc:407] Connection header received
0807 15:07:06.457972 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.457973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.457994 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.458022 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.458629 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.459315 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.460611 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.460883 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.460940 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.462281 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.462282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.462282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.462289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.463946 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.464302 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.464303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.464303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.464361 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.464533 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.464534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.464535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.464644 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.464733 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.464842 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.464913 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:07:06.467182 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:06.460828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.18.129.61:35837)
0807 15:07:06.460867 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:06.460869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:06.460902 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:07:06.460960 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:06.460961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:06.460973 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:06.461052 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:06.461592 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.462311 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.463066 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:06.463327 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:06.463388 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:06.464484 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:06.464485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:06.464485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:06.464493 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:06.466097 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.466386 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.466387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.466387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.466449 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:06.466615 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:06.466616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:06.466616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:06.466744 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:06.466834 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:06.466979 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 15:07:06.467069 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:07:07.435961 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.435667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.1:41299)
0807 15:07:07.435804 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.435896 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260807 15:07:07.436120 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.435651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.3:33733)
0807 15:07:07.435812 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.436006 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
W20260807 15:07:07.436228 29462 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:46499 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
W20260807 15:07:07.436324 29895 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:46499 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
I20260807 15:07:07.436771 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.436550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.2:50037)
0807 15:07:07.436650 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.436730 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":51,"threads_started":1}
W20260807 15:07:07.436957 29690 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:46499 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
I20260807 15:07:07.939977 29996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0d381b52ca30418db8937a2848edd56a)
I20260807 15:07:07.940122 29996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } }
I20260807 15:07:07.940443 29996 leader_election.cc:288] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d381b52ca30418db8937a2848edd56a (127.18.129.62:46499), 2a8d99b921f5477889667008c53b93ed (127.18.129.60:44647)
I20260807 15:07:07.940717 29999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0d381b52ca30418db8937a2848edd56a)
I20260807 15:07:07.941063 29999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } }
I20260807 15:07:07.941128 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.940479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:35020)
0807 15:07:07.940618 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.941086 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":46,"threads_started":1}
W20260807 15:07:07.941243 28882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
I20260807 15:07:07.941495 29999 leader_election.cc:288] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d381b52ca30418db8937a2848edd56a (127.18.129.62:46499), 76d63311c9c9410c840cca74d203c16b (127.18.129.61:35837)
I20260807 15:07:07.941618 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.941535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:35024)
0807 15:07:07.941571 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.941595 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 15:07:07.941699 28882 leader_election.cc:334] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0d381b52ca30418db8937a2848edd56a (127.18.129.62:46499): Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
I20260807 15:07:07.941882 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.941600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:35030)
0807 15:07:07.941802 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.941847 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":56,"threads_started":1}
W20260807 15:07:07.941989 29162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
I20260807 15:07:07.942423 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.942309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:35044)
0807 15:07:07.942366 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.942397 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 15:07:07.942515 29162 leader_election.cc:334] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0d381b52ca30418db8937a2848edd56a (127.18.129.62:46499): Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
I20260807 15:07:07.943992 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.940796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45084 (local address 127.18.129.60:44647)
0807 15:07:07.941166 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:07.941168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:07.941176 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:07.941198 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:07.941199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:07.941209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:07.941265 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:07.941424 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:07.942187 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:07.943325 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:07.943704 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:07.943763 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:07.943774 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:07.943896 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 15:07:07.943935 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":63,"threads_started":1}
I20260807 15:07:07.943992 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.940590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:44647 (local address 127.0.0.1:45084)
0807 15:07:07.940706 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.940717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:07.940772 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:07.941270 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:07.941271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:07.941273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:07.941339 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:07.941341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:07.942278 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:07.942279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:07.943249 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:07.943252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:07.943773 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:07.943774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:07.943833 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:07.943836 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:07.943883 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:07:07.943922 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":1,"thread_start_us":49,"threads_started":1}
I20260807 15:07:07.944239 29197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76d63311c9c9410c840cca74d203c16b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2a8d99b921f5477889667008c53b93ed" is_pre_election: true
I20260807 15:07:07.944329 29197 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76d63311c9c9410c840cca74d203c16b in term 1.
I20260807 15:07:07.944460 28877 leader_election.cc:302] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a8d99b921f5477889667008c53b93ed, 76d63311c9c9410c840cca74d203c16b; no voters: 0d381b52ca30418db8937a2848edd56a
I20260807 15:07:07.944603 29996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:07:07.944641 29996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0d381b52ca30418db8937a2848edd56a)
I20260807 15:07:07.944661 29996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:07:07.944788 30006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.941755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35837 (local address 127.0.0.1:53676)
0807 15:07:07.941916 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.942168 (+ 252us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:07.942229 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:07.942386 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:07.942386 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:07.942389 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:07.942435 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:07.942436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:07.943142 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:07.943143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:07.943898 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:07.943900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:07.944564 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:07.944565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:07.944631 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:07.944634 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:07.944678 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:07:07.944719 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":36,"threads_started":1}
I20260807 15:07:07.944801 30007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.941879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.18.129.61:35837)
0807 15:07:07.942015 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:07.942017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:07.942233 (+ 216us) server_negotiation.cc:407] Connection header received
0807 15:07:07.942287 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:07.942288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:07.942298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:07.942360 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:07.942540 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:07.943070 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:07.943978 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:07.944482 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:07.944544 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:07.944555 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:07.944688 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 15:07:07.944721 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":113,"thread_start_us":93,"threads_started":1}
I20260807 15:07:07.945078 28919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a8d99b921f5477889667008c53b93ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "76d63311c9c9410c840cca74d203c16b" is_pre_election: true
I20260807 15:07:07.945545 29996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } }
I20260807 15:07:07.945679 29996 leader_election.cc:288] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [CANDIDATE]: Term 2 election: Requested vote from peers 0d381b52ca30418db8937a2848edd56a (127.18.129.62:46499), 2a8d99b921f5477889667008c53b93ed (127.18.129.60:44647)
I20260807 15:07:07.945676 28919 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a8d99b921f5477889667008c53b93ed in current term 2: Already voted for candidate 76d63311c9c9410c840cca74d203c16b in this term.
I20260807 15:07:07.945827 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.945743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:35054)
0807 15:07:07.945775 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.945804 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 15:07:07.945873 29159 leader_election.cc:302] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a8d99b921f5477889667008c53b93ed; no voters: 0d381b52ca30418db8937a2848edd56a, 76d63311c9c9410c840cca74d203c16b
I20260807 15:07:07.945874 29197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76d63311c9c9410c840cca74d203c16b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2a8d99b921f5477889667008c53b93ed"
I20260807 15:07:07.945986 29197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:07:07.946141 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.946028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:35058)
0807 15:07:07.946092 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.946120 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 15:07:07.946237 28882 leader_election.cc:334] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0d381b52ca30418db8937a2848edd56a (127.18.129.62:46499): Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
I20260807 15:07:07.946620 29197 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76d63311c9c9410c840cca74d203c16b in term 2.
I20260807 15:07:07.946693 29999 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:07:07.946766 28877 leader_election.cc:302] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a8d99b921f5477889667008c53b93ed, 76d63311c9c9410c840cca74d203c16b; no voters: 0d381b52ca30418db8937a2848edd56a
I20260807 15:07:07.946838 29996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:07:07.946991 29996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [term 2 LEADER]: Becoming Leader. State: Replica: 76d63311c9c9410c840cca74d203c16b, State: Running, Role: LEADER
I20260807 15:07:07.947057 29996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } }
I20260807 15:07:07.947350 30009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76d63311c9c9410c840cca74d203c16b. Latest consensus state: current_term: 2 leader_uuid: "76d63311c9c9410c840cca74d203c16b" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:07.947445 30009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:07.947636 30013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:07.948112 30013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:07.948186 30013 catalog_manager.cc:1259] Loaded cluster ID: 37779a6cc96443ad963ac6ff683842dd
I20260807 15:07:07.948217 30013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:07.948374 30013 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:07.948464 30013 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Loaded TSK: 0
I20260807 15:07:07.948604 30013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:07:07.996363 28900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:07:07.997931 28900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:07.998941 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.998807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:35074)
0807 15:07:07.998874 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.998910 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 15:07:07.998945 29197 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [term 2 FOLLOWER]: Refusing update from remote peer 76d63311c9c9410c840cca74d203c16b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 15:07:07.999259 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:07.999173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.0.0.1:35086)
0807 15:07:07.999207 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:07.999236 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 15:07:07.999280 29996 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 15:07:07.999403 28882 consensus_peers.cc:487] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b -> Peer 0d381b52ca30418db8937a2848edd56a (127.18.129.62:46499): Couldn't send request to peer 0d381b52ca30418db8937a2848edd56a. Status: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:07:07.999897 29999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76d63311c9c9410c840cca74d203c16b. Latest consensus state: current_term: 2 leader_uuid: "76d63311c9c9410c840cca74d203c16b" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:07.999991 29999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:07:08.000329 30014 mvcc.cc:205] Tried to move back new op lower bound from 7315927973881876480 to 7315927973671620608. Current Snapshot: MvccSnapshot[applied={T|T < 7315927973881876480}]
I20260807 15:07:08.000434 30009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "76d63311c9c9410c840cca74d203c16b" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:08.000521 30009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:08.000669 29999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "76d63311c9c9410c840cca74d203c16b" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "0d381b52ca30418db8937a2848edd56a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46499 } } peers { permanent_uuid: "76d63311c9c9410c840cca74d203c16b" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35837 } attrs { promote: false } } peers { permanent_uuid: "2a8d99b921f5477889667008c53b93ed" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 44647 } attrs { promote: false } } }
I20260807 15:07:08.000761 29999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8d99b921f5477889667008c53b93ed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:07:08.005237 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.002224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43269 (local address 127.0.0.1:54060)
0807 15:07:08.002478 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.002488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:08.002594 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:08.002752 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:08.002753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:08.002754 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:08.002781 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:08.002782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.003667 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.003668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:08.004362 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:08.004364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.005010 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.005011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:08.005076 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:08.005081 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:07:08.005126 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:07:08.005167 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":71,"threads_started":1}
I20260807 15:07:08.005239 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.002392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54060 (local address 127.18.129.2:43269)
0807 15:07:08.002501 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:08.002504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:08.002536 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:07:08.002667 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:08.002667 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:08.002678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:08.002832 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:08.002897 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.003610 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.004446 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.004941 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.004998 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:08.005011 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:08.005137 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:07:08.005171 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260807 15:07:08.005658 30019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.002210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47642 (local address 127.18.129.1:36789)
0807 15:07:08.002497 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:08.002499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:08.002506 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:07:08.002529 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:08.002530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:08.002540 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:08.002597 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:08.002746 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.003534 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.004730 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.005288 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.005391 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:08.005407 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:08.005544 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:07:08.005591 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":68,"threads_started":1}
I20260807 15:07:08.005677 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.002145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36789 (local address 127.0.0.1:47642)
0807 15:07:08.002216 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.002227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:08.002285 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:08.002602 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:08.002603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:08.002606 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:08.002661 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:08.002662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.003629 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.003630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:08.004645 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:08.004647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.005397 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.005398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:08.005459 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:08.005462 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:08.005523 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:07:08.005564 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 15:07:08.005620 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.002181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:46147 (local address 127.0.0.1:51530)
0807 15:07:08.002367 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.002374 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:08.002408 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:08.002667 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:08.002668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:08.002671 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:08.002715 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:08.002716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.003391 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.003392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:08.004559 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:08.004562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.005390 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.005391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:08.005453 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:08.005457 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:07:08.005486 (+ 29us) client_negotiation.cc:240] Negotiation successful
0807 15:07:08.005521 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":1}
I20260807 15:07:08.005712 29587 tablet_service.cc:1467] Processing CreateTablet for tablet d6bbcea893ca49a6945e64f7e84c4e7e (DEFAULT_TABLE table=test-table [id=db9337de6c1547f08b85fd01c28c9729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:08.005867 29587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6bbcea893ca49a6945e64f7e84c4e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:08.005905 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.002278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51530 (local address 127.18.129.3:46147)
0807 15:07:08.002401 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:08.002403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:08.002411 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:08.002468 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:08.002469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:08.002480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:08.002777 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:08.002817 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.003318 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.004752 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.005305 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.005779 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:08.005796 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:08.005814 (+ 18us) server_negotiation.cc:299] Negotiation successful
0807 15:07:08.005853 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":76,"threads_started":1}
I20260807 15:07:08.006031 29343 tablet_service.cc:1467] Processing CreateTablet for tablet d6bbcea893ca49a6945e64f7e84c4e7e (DEFAULT_TABLE table=test-table [id=db9337de6c1547f08b85fd01c28c9729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:08.006143 29343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6bbcea893ca49a6945e64f7e84c4e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:08.006479 29797 tablet_service.cc:1467] Processing CreateTablet for tablet d6bbcea893ca49a6945e64f7e84c4e7e (DEFAULT_TABLE table=test-table [id=db9337de6c1547f08b85fd01c28c9729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:08.006590 29797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6bbcea893ca49a6945e64f7e84c4e7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:08.007440 30028 tablet_bootstrap.cc:492] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: Bootstrap starting.
I20260807 15:07:08.007624 30029 tablet_bootstrap.cc:492] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d: Bootstrap starting.
I20260807 15:07:08.007803 30028 tablet_bootstrap.cc:654] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:08.007958 30029 tablet_bootstrap.cc:654] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:08.008118 30030 tablet_bootstrap.cc:492] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: Bootstrap starting.
I20260807 15:07:08.008469 30030 tablet_bootstrap.cc:654] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:08.008541 30028 tablet_bootstrap.cc:492] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: No bootstrap required, opened a new log
I20260807 15:07:08.008599 30028 ts_tablet_manager.cc:1397] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:08.008765 30028 raft_consensus.cc:348] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.008846 30028 raft_consensus.cc:374] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:08.008868 30028 raft_consensus.cc:729] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca0652f5f53478ea3dd9ac1fd968025, State: Initialized, Role: FOLLOWER
I20260807 15:07:08.008919 30028 consensus_queue.cc:260] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.008970 30030 tablet_bootstrap.cc:492] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: No bootstrap required, opened a new log
I20260807 15:07:08.009016 30030 ts_tablet_manager.cc:1397] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:08.009069 30028 ts_tablet_manager.cc:1428] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:08.009135 30030 raft_consensus.cc:348] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.009196 30030 raft_consensus.cc:374] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:08.009214 30030 raft_consensus.cc:729] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d97875043184639b13b66f88a153466, State: Initialized, Role: FOLLOWER
I20260807 15:07:08.009249 30030 consensus_queue.cc:260] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.009372 30030 ts_tablet_manager.cc:1428] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:08.009651 29691 heartbeater.cc:503] Master 127.18.129.61:35837 was elected leader, sending a full tablet report...
I20260807 15:07:08.010111 30029 tablet_bootstrap.cc:492] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d: No bootstrap required, opened a new log
I20260807 15:07:08.010167 30029 ts_tablet_manager.cc:1397] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:07:08.010296 30029 raft_consensus.cc:348] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.010352 30029 raft_consensus.cc:374] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:08.010365 30029 raft_consensus.cc:729] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff6d3654b92431fb6c63e8f63b41a2d, State: Initialized, Role: FOLLOWER
I20260807 15:07:08.010372 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.009238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.2:34295)
0807 15:07:08.010305 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.010339 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1041,"thread_start_us":80,"threads_started":1}
I20260807 15:07:08.010414 30029 consensus_queue.cc:260] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.010568 30029 ts_tablet_manager.cc:1428] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:08.010798 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.010599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.2:56811)
0807 15:07:08.010724 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.010765 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":82,"threads_started":1}
I20260807 15:07:08.011039 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.010808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.1:32865)
0807 15:07:08.010960 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.010996 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":104,"threads_started":1}
I20260807 15:07:08.011219 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.009571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.3:50305)
0807 15:07:08.010503 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.011181 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":79,"threads_started":1}
I20260807 15:07:08.011415 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.011267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.1:38931)
0807 15:07:08.011364 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.011390 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 15:07:08.011550 29897 heartbeater.cc:503] Master 127.18.129.61:35837 was elected leader, sending a full tablet report...
I20260807 15:07:08.011603 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.011501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.3:47581)
0807 15:07:08.011546 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.011575 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 15:07:08.046927 30047 raft_consensus.cc:482] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:08.047022 30047 raft_consensus.cc:504] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.047155 30047 leader_election.cc:288] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789), 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269)
I20260807 15:07:08.047386 29612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6bbcea893ca49a6945e64f7e84c4e7e" candidate_uuid: "7d97875043184639b13b66f88a153466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" is_pre_election: true
I20260807 15:07:08.047374 29384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6bbcea893ca49a6945e64f7e84c4e7e" candidate_uuid: "7d97875043184639b13b66f88a153466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" is_pre_election: true
I20260807 15:07:08.047480 29612 raft_consensus.cc:2454] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d97875043184639b13b66f88a153466 in term 0.
I20260807 15:07:08.047483 29384 raft_consensus.cc:2454] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d97875043184639b13b66f88a153466 in term 0.
I20260807 15:07:08.047644 29721 leader_election.cc:302] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ff6d3654b92431fb6c63e8f63b41a2d, 7d97875043184639b13b66f88a153466; no voters:
I20260807 15:07:08.047747 30047 raft_consensus.cc:2781] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:08.047788 30047 raft_consensus.cc:482] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:08.047807 30047 raft_consensus.cc:3037] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:08.048501 30047 raft_consensus.cc:504] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.048621 30047 leader_election.cc:288] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [CANDIDATE]: Term 1 election: Requested vote from peers 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789), 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269)
I20260807 15:07:08.048795 29384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6bbcea893ca49a6945e64f7e84c4e7e" candidate_uuid: "7d97875043184639b13b66f88a153466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d"
I20260807 15:07:08.048831 29612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6bbcea893ca49a6945e64f7e84c4e7e" candidate_uuid: "7d97875043184639b13b66f88a153466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca0652f5f53478ea3dd9ac1fd968025"
I20260807 15:07:08.048873 29384 raft_consensus.cc:3037] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:08.048894 29612 raft_consensus.cc:3037] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:08.049480 29384 raft_consensus.cc:2454] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d97875043184639b13b66f88a153466 in term 1.
I20260807 15:07:08.049540 29612 raft_consensus.cc:2454] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d97875043184639b13b66f88a153466 in term 1.
I20260807 15:07:08.049690 29721 leader_election.cc:302] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca0652f5f53478ea3dd9ac1fd968025, 7d97875043184639b13b66f88a153466; no voters:
I20260807 15:07:08.049791 30047 raft_consensus.cc:2781] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:08.049954 30047 raft_consensus.cc:686] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 1 LEADER]: Becoming Leader. State: Replica: 7d97875043184639b13b66f88a153466, State: Running, Role: LEADER
I20260807 15:07:08.050014 30047 consensus_queue.cc:237] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } }
I20260807 15:07:08.050345 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.050224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.3:43717)
0807 15:07:08.050267 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.050306 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 15:07:08.050619 28900 catalog_manager.cc:5495] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d97875043184639b13b66f88a153466 (127.18.129.3). New cstate: current_term: 1 leader_uuid: "7d97875043184639b13b66f88a153466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d97875043184639b13b66f88a153466" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 46147 } health_report { overall_health: HEALTHY } } }
I20260807 15:07:08.062105 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.059951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51540 (local address 127.18.129.3:46147)
0807 15:07:08.060008 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:08.060010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:08.060018 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:08.060058 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:08.060058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:08.060067 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:08.060110 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:08.060242 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.060770 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.061489 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:08.061686 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:08.061743 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:08.061869 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:08.061901 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:08.061940 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:08.061994 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:07:08.062017 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:07:08.062948 29612 raft_consensus.cc:1261] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 1 FOLLOWER]: Refusing update from remote peer 7d97875043184639b13b66f88a153466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:08.062978 29384 raft_consensus.cc:1261] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 1 FOLLOWER]: Refusing update from remote peer 7d97875043184639b13b66f88a153466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:08.063200 30056 consensus_queue.cc:1035] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ff6d3654b92431fb6c63e8f63b41a2d" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:08.063318 30047 consensus_queue.cc:1035] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca0652f5f53478ea3dd9ac1fd968025" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:08.063597 29463 heartbeater.cc:503] Master 127.18.129.61:35837 was elected leader, sending a full tablet report...
I20260807 15:07:08.063839 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.063727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.1:57043)
0807 15:07:08.063771 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.063802 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 15:07:08.064038 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.063929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.2:53225)
0807 15:07:08.063972 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.064008 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 15:07:08.065052 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.064948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.3:47173)
0807 15:07:08.064984 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.065012 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 15:07:08.065099 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.065003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.1:48615)
0807 15:07:08.065036 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.065059 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260807 15:07:08.066349 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.066259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.2:40383)
0807 15:07:08.066297 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.066320 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 15:07:08.066634 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.065768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46499 (local address 127.18.129.3:32923)
0807 15:07:08.066574 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.066604 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:46499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":788}
I20260807 15:07:08.066730 28902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:08.066821 28902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:08.067973 28902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76d63311c9c9410c840cca74d203c16b: Sending DeleteTablet for 3 replicas of tablet d6bbcea893ca49a6945e64f7e84c4e7e
I20260807 15:07:08.068279 29797 tablet_service.cc:1514] Processing DeleteTablet for tablet d6bbcea893ca49a6945e64f7e84c4e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:08 UTC) from {username='oryx', principal='oryx/127.18.129.61@KRBTEST.COM'} at 127.0.0.1:51530
I20260807 15:07:08.068312 29343 tablet_service.cc:1514] Processing DeleteTablet for tablet d6bbcea893ca49a6945e64f7e84c4e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:08 UTC) from {username='oryx', principal='oryx/127.18.129.61@KRBTEST.COM'} at 127.0.0.1:47642
I20260807 15:07:08.068461 30072 tablet_replica.cc:330] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: stopping tablet replica
I20260807 15:07:08.068519 30075 tablet_replica.cc:330] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d: stopping tablet replica
I20260807 15:07:08.068558 29587 tablet_service.cc:1514] Processing DeleteTablet for tablet d6bbcea893ca49a6945e64f7e84c4e7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:08 UTC) from {username='oryx', principal='oryx/127.18.129.61@KRBTEST.COM'} at 127.0.0.1:54060
I20260807 15:07:08.068544 30072 raft_consensus.cc:2229] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:07:08.068589 30075 raft_consensus.cc:2229] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:08.068634 30075 raft_consensus.cc:2258] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:08.068698 30072 raft_consensus.cc:2258] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:08.068892 30076 tablet_replica.cc:330] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: stopping tablet replica
I20260807 15:07:08.068943 30076 raft_consensus.cc:2229] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:08.068977 30076 raft_consensus.cc:2258] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:08.069020 30075 ts_tablet_manager.cc:1905] T d6bbcea893ca49a6945e64f7e84c4e7e P 4ff6d3654b92431fb6c63e8f63b41a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:08.069180 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 29122
I20260807 15:07:08.077440 30072 ts_tablet_manager.cc:1905] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:08.079046 30072 ts_tablet_manager.cc:1918] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:08.079124 30072 log.cc:1196] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/d6bbcea893ca49a6945e64f7e84c4e7e
I20260807 15:07:08.079281 30076 ts_tablet_manager.cc:1905] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:08.079357 30072 ts_tablet_manager.cc:1939] T d6bbcea893ca49a6945e64f7e84c4e7e P 7d97875043184639b13b66f88a153466: Deleting consensus metadata
I20260807 15:07:08.079699 28877 catalog_manager.cc:4841] TS 7d97875043184639b13b66f88a153466 (127.18.129.3:46147): tablet d6bbcea893ca49a6945e64f7e84c4e7e (table test-table [id=db9337de6c1547f08b85fd01c28c9729]) successfully deleted
I20260807 15:07:08.080595 30076 ts_tablet_manager.cc:1918] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:08.080646 30076 log.cc:1196] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/d6bbcea893ca49a6945e64f7e84c4e7e
I20260807 15:07:08.080824 30076 ts_tablet_manager.cc:1939] T d6bbcea893ca49a6945e64f7e84c4e7e P 1ca0652f5f53478ea3dd9ac1fd968025: Deleting consensus metadata
I20260807 15:07:08.081133 28878 catalog_manager.cc:4841] TS 1ca0652f5f53478ea3dd9ac1fd968025 (127.18.129.2:43269): tablet d6bbcea893ca49a6945e64f7e84c4e7e (table test-table [id=db9337de6c1547f08b85fd01c28c9729]) successfully deleted
I20260807 15:07:08.081792 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:08.081681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36789 (local address 127.0.0.1:47646)
0807 15:07:08.081728 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:08.081759 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:36789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:07:08.081960 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 29472
W20260807 15:07:08.081959 28878 catalog_manager.cc:4584] TS 4ff6d3654b92431fb6c63e8f63b41a2d (127.18.129.1:36789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6bbcea893ca49a6945e64f7e84c4e7e: Network error: Client connection negotiation failed: client connection to 127.18.129.1:36789: connect: Connection refused (error 111)
I20260807 15:07:08.090579 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 29702
I20260807 15:07:08.098031 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 28697
I20260807 15:07:08.108103 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 28988
2026-08-07T15:07:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4818 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:07:08 dist-test-slave-5bn2 krb5kdc[30081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:07:08 dist-test-slave-5bn2 krb5kdc[30081](info): set up 2 sockets
Aug 07 15:07:08 dist-test-slave-5bn2 krb5kdc[30081](info): commencing operation
krb5kdc: starting...
W20260807 15:07:09.243584 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.62.keytab.
Entry for principal oryx/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.62.keytab.
I20260807 15:07:09.362964 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:35845
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.62.keytab
--principal=oryx/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:35845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:09.444727 30101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:09.444923 30101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:09.444943 30101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:09.446074 30101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:09.446115 30101 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:07:09.446127 30101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:09.446142 30101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:09.446167 30101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:09.446189 30101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:09.447759 30101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:35845
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.62.keytab
--principal=oryx/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:35845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:09.447991 30101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:09.448216 30101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:07:09.453053 30101 server_base.cc:1034] running on GCE node
W20260807 15:07:09.453037 30117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:09.453025 30109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:09.453011 30112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:09.453504 30101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:09.453868 30101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:09.454998 30101 hybrid_clock.cc:648] HybridClock initialized: now 1786115229454981 us; error 39 us; skew 500 ppm
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:09.457436 30101 init.cc:377] Logged in from keytab as oryx/127.18.129.62@KRBTEST.COM (short username oryx)
I20260807 15:07:09.457875 30101 webserver.cc:460] Webserver started at http://127.18.129.62:32805/ using document root <none> and password file <none>
I20260807 15:07:09.458102 30101 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:09.458148 30101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:09.458256 30101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:09.459316 30101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "350ec1582ccb4162b9319d7e50547434"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "0825fbff7c3b829325924a72bd6b8f3d"
server_key_iv: "ee3f177144b47b3261b4e85c26b6c545"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.459657 30101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "350ec1582ccb4162b9319d7e50547434"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "0825fbff7c3b829325924a72bd6b8f3d"
server_key_iv: "ee3f177144b47b3261b4e85c26b6c545"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.460932 30101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:07:09.461673 30125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:09.461844 30101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:09.461935 30101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "350ec1582ccb4162b9319d7e50547434"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "0825fbff7c3b829325924a72bd6b8f3d"
server_key_iv: "ee3f177144b47b3261b4e85c26b6c545"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.461997 30101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:09.469748 30101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:09.471712 30101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:09.471849 30101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:09.476442 30101 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:35845
I20260807 15:07:09.476496 30227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:35845 every 8 connection(s)
I20260807 15:07:09.476838 30101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:09.477512 30228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:09.477757 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 30101
I20260807 15:07:09.477849 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:09.478024 18948 external_mini_cluster.cc:1428] Setting key 220fd1d55611a8b90fb860589741a517
I20260807 15:07:09.480986 30228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: Bootstrap starting.
I20260807 15:07:09.481686 30228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:09.481959 30228 log.cc:824] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:09.482661 30228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: No bootstrap required, opened a new log
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:09.484730 30228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350ec1582ccb4162b9319d7e50547434" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35845 } }
I20260807 15:07:09.484898 30228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:09.484925 30228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350ec1582ccb4162b9319d7e50547434, State: Initialized, Role: FOLLOWER
I20260807 15:07:09.485049 30228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350ec1582ccb4162b9319d7e50547434" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35845 } }
I20260807 15:07:09.485116 30228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:09.485146 30228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:09.485193 30228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:09.485827 30228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350ec1582ccb4162b9319d7e50547434" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35845 } }
I20260807 15:07:09.485931 30228 leader_election.cc:302] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350ec1582ccb4162b9319d7e50547434; no voters:
I20260807 15:07:09.486093 30228 leader_election.cc:288] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:09.486210 30239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:09.486332 30228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:09.486433 30239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [term 1 LEADER]: Becoming Leader. State: Replica: 350ec1582ccb4162b9319d7e50547434, State: Running, Role: LEADER
I20260807 15:07:09.486559 30239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350ec1582ccb4162b9319d7e50547434" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35845 } }
I20260807 15:07:09.486930 30240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "350ec1582ccb4162b9319d7e50547434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350ec1582ccb4162b9319d7e50547434" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35845 } } }
I20260807 15:07:09.487005 30240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:09.487138 30240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350ec1582ccb4162b9319d7e50547434. Latest consensus state: current_term: 1 leader_uuid: "350ec1582ccb4162b9319d7e50547434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350ec1582ccb4162b9319d7e50547434" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35845 } } }
I20260807 15:07:09.487186 30240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:09.487247 30254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:09.487910 30254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:09.489092 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.478530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.18.129.62:35845)
0807 15:07:09.478730 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:09.478740 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:09.478773 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:07:09.479285 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:09.479295 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:09.479360 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:09.479505 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:09.480089 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.481478 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.482251 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.482730 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.482842 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.485164 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:09.485169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:09.485169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:09.485182 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:09.487095 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.487491 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.487494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.487511 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.487615 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.487840 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.487841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.487842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.488170 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:09.488324 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:09.488744 (+ 420us) server_negotiation.cc:299] Negotiation successful
0807 15:07:09.488890 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260807 15:07:09.489435 30254 catalog_manager.cc:1347] Generated new cluster ID: 188bd5b265ea49fe87f0bb0ef28f023f
I20260807 15:07:09.489486 30254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:09.495398 30254 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:09.496085 30254 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:09.500353 30254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: Generated new TSK 0
I20260807 15:07:09.500514 30254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab.
Entry for principal oryx/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab.
I20260807 15:07:09.535048 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35845
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab
--principal=oryx/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:09.630143 30268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:09.630365 30268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:09.630399 30268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:09.631567 30268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:09.631613 30268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:09.631640 30268 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:07:09.631664 30268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:09.633188 30268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab
--principal=oryx/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:09.633420 30268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:09.633626 30268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:09.638691 30279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:09.638867 30268 server_base.cc:1034] running on GCE node
W20260807 15:07:09.638734 30283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:09.638680 30278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:09.639174 30268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:09.639401 30268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:09.640527 30268 hybrid_clock.cc:648] HybridClock initialized: now 1786115229640521 us; error 37 us; skew 500 ppm
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:09.643006 30268 init.cc:377] Logged in from keytab as oryx/127.18.129.1@KRBTEST.COM (short username oryx)
I20260807 15:07:09.643422 30268 webserver.cc:460] Webserver started at http://127.18.129.1:35315/ using document root <none> and password file <none>
I20260807 15:07:09.643606 30268 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:09.643646 30268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:09.643738 30268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:09.644716 30268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "f51cc4513040492d816536dc9db09fed"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "e20c8e0c46002031693adaf88a47405f"
server_key_iv: "79387625d74ea7f90ce7de631be6bd04"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.645008 30268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "f51cc4513040492d816536dc9db09fed"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "e20c8e0c46002031693adaf88a47405f"
server_key_iv: "79387625d74ea7f90ce7de631be6bd04"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.646188 30268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:07:09.646799 30291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:09.646965 30268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:09.647053 30268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "f51cc4513040492d816536dc9db09fed"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "e20c8e0c46002031693adaf88a47405f"
server_key_iv: "79387625d74ea7f90ce7de631be6bd04"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.647122 30268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:09.667562 30268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:09.669736 30268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:09.669880 30268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:09.670444 30268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:09.670495 30268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:09.670526 30268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:09.670542 30268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:09.677341 30268 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:35383
I20260807 15:07:09.677439 30472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:35383 every 8 connection(s)
I20260807 15:07:09.677736 30268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:07:09.679661 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.670806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:42839 (local address 127.18.129.62:35845)
0807 15:07:09.670893 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:09.670895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:09.671176 (+ 281us) server_negotiation.cc:407] Connection header received
0807 15:07:09.671687 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:09.671688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:09.671700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:09.671755 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:09.672410 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.672924 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.673960 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.674170 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.674227 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.677106 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:09.677108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:09.677108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:09.677117 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:09.678558 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.678956 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.678957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.678958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.679010 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.679191 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.679191 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.679192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.679290 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:09.679335 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:09.679529 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 15:07:09.679596 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:07:09.679816 30304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.670791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35845 (local address 127.18.129.1:42839)
0807 15:07:09.671065 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:09.671095 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:09.671587 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:09.671830 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:09.671834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:09.672127 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:09.672315 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.672320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.673038 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.673039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.673879 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.673881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.674247 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.674248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.674359 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.674828 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:09.674839 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:09.677010 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:09.678641 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.678648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.678649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.678890 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.679052 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.679053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.679053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.679145 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.679332 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:09.679334 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:09.679441 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 15:07:09.679510 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:07:09.679632 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260807 15:07:09.680305 30474 heartbeater.cc:347] Connected to a master server at 127.18.129.62:35845
I20260807 15:07:09.680387 30474 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:09.680559 30474 heartbeater.cc:511] Master 127.18.129.62:35845 requested a full tablet report, sending...
I20260807 15:07:09.681118 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 30268
I20260807 15:07:09.681123 30144 ts_manager.cc:194] Registered new tserver with Master: f51cc4513040492d816536dc9db09fed (127.18.129.1:35383)
I20260807 15:07:09.681221 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:07:09.681440 18948 external_mini_cluster.cc:1428] Setting key c826a4266c2a0a1b4310f0d2a06d6a75
I20260807 15:07:09.681962 30144 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:42839
I20260807 15:07:09.686705 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.681409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:36877 (local address 127.18.129.62:35845)
0807 15:07:09.681450 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:09.681451 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:09.681587 (+ 136us) server_negotiation.cc:407] Connection header received
0807 15:07:09.681759 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:09.681760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:09.681770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:09.681826 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:09.682189 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.682634 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.683358 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.683575 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.683641 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.684440 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:09.684441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:09.684442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:09.684448 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:09.685696 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.685993 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.685995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.685995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.686049 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.686235 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.686236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.686236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.686358 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:09.686417 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:09.686547 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 15:07:09.686615 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:07:09.686721 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.681353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35845 (local address 127.18.129.1:36877)
0807 15:07:09.681509 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:09.681519 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:09.681682 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:09.681829 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:09.681829 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:09.682027 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:09.682096 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.682098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.682716 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.682716 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.683286 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.683287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.683663 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.683664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.683750 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.684114 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:09.684118 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:09.684363 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:09.685772 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.685773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.685773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.685934 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.686101 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.686102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.686102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.686160 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.686409 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:09.686410 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:09.686457 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 15:07:09.686518 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:07:09.686617 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab.
Entry for principal oryx/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab.
I20260807 15:07:09.707172 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35845
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab
--principal=oryx/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:09.788283 30487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:09.788492 30487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:09.788515 30487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:09.789613 30487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:09.789649 30487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:09.789667 30487 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:07:09.789688 30487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:09.791200 30487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab
--principal=oryx/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:09.791386 30487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:09.791600 30487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:09.796541 30496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:09.796518 30493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:09.796556 30487 server_base.cc:1034] running on GCE node
W20260807 15:07:09.796526 30494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:09.796907 30487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:09.797143 30487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:09.798283 30487 hybrid_clock.cc:648] HybridClock initialized: now 1786115229798275 us; error 34 us; skew 500 ppm
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:09.800724 30487 init.cc:377] Logged in from keytab as oryx/127.18.129.2@KRBTEST.COM (short username oryx)
I20260807 15:07:09.801123 30487 webserver.cc:460] Webserver started at http://127.18.129.2:35183/ using document root <none> and password file <none>
I20260807 15:07:09.801298 30487 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:09.801337 30487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:09.801431 30487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:09.802412 30487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "7ee6853fb1f44f21afaa25e67072871c"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "661c402b029a72d2132ab95b7625a481"
server_key_iv: "d1da2716d97635de716f02db2cd4c11f"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.802708 30487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "7ee6853fb1f44f21afaa25e67072871c"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "661c402b029a72d2132ab95b7625a481"
server_key_iv: "d1da2716d97635de716f02db2cd4c11f"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.804240 30487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 15:07:09.805088 30517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:09.805276 30487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:09.805364 30487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "7ee6853fb1f44f21afaa25e67072871c"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "661c402b029a72d2132ab95b7625a481"
server_key_iv: "d1da2716d97635de716f02db2cd4c11f"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.805434 30487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:09.812836 30487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:09.814834 30487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:09.815013 30487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:09.815513 30487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:09.815552 30487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:09.815590 30487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:09.815614 30487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:09.822431 30487 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43023
I20260807 15:07:09.822525 30697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43023 every 8 connection(s)
I20260807 15:07:09.822825 30487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:07:09.825237 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.815909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:56753 (local address 127.18.129.62:35845)
0807 15:07:09.815952 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:09.815954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:09.816182 (+ 228us) server_negotiation.cc:407] Connection header received
0807 15:07:09.816682 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:09.816684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:09.816697 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:09.816749 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:09.817423 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.818182 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.819716 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.820023 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.820099 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.822819 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:09.822821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:09.822821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:09.822830 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:09.824120 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.824475 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.824476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.824476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.824530 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.824736 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.824737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.824737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.824837 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:09.824886 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:09.825106 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 15:07:09.825172 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:07:09.825418 30532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.815913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35845 (local address 127.18.129.2:56753)
0807 15:07:09.816067 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:09.816102 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:09.816586 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:09.816820 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:09.816824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:09.817143 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:09.817328 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.817333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.818283 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.818284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.819631 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.819636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.820130 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.820132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.820280 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.820850 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:09.820861 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:09.822732 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:09.824190 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.824195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.824197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.824413 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.824592 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.824592 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.824593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.824690 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.824883 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:09.824886 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:09.825000 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:07:09.825089 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:07:09.825224 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260807 15:07:09.825888 30701 heartbeater.cc:347] Connected to a master server at 127.18.129.62:35845
I20260807 15:07:09.825973 30701 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:09.826143 30701 heartbeater.cc:511] Master 127.18.129.62:35845 requested a full tablet report, sending...
I20260807 15:07:09.826558 30146 ts_manager.cc:194] Registered new tserver with Master: 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023)
I20260807 15:07:09.827028 30146 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:56753
I20260807 15:07:09.832198 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 30487
I20260807 15:07:09.832316 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:07:09.832362 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.826985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:39167 (local address 127.18.129.62:35845)
0807 15:07:09.827023 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:09.827025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:09.827155 (+ 130us) server_negotiation.cc:407] Connection header received
0807 15:07:09.827199 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:09.827200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:09.827210 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:09.827261 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:09.827548 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.828052 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.828873 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.829067 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.829125 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.829985 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:09.829987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:09.829987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:09.829994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:09.831281 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.831602 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.831603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.831604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.831658 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.831819 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.831820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.831821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.831966 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:09.832027 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:09.832193 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 15:07:09.832263 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:07:09.832373 30709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.826937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35845 (local address 127.18.129.2:39167)
0807 15:07:09.827078 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:09.827089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:09.827153 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:09.827262 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:09.827263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:09.827422 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:09.827478 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.827479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.828148 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.828149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.828796 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.828798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.829130 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.829131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.829202 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.829610 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:09.829616 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:09.829903 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:09.831358 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.831359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.831359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.831526 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.831711 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.831712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.831712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.831772 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.832047 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:09.832048 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:09.832106 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 15:07:09.832160 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:07:09.832265 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260807 15:07:09.832518 18948 external_mini_cluster.cc:1428] Setting key 4c366a0128b058f8390093715c0f8eab
WARNING: no policy specified for oryx/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab.
Entry for principal oryx/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab.
I20260807 15:07:09.859614 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35845
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab
--principal=oryx/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:09.944962 30716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:09.945171 30716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:09.945195 30716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:09.946359 30716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:09.946411 30716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:09.946430 30716 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:07:09.946452 30716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:09.947965 30716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab
--principal=oryx/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.129.62:35845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:09.948158 30716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:09.948343 30716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:09.953559 30728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:09.953545 30729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:09.953562 30716 server_base.cc:1034] running on GCE node
W20260807 15:07:09.953539 30732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:09.953986 30716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:09.954212 30716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:09.955360 30716 hybrid_clock.cc:648] HybridClock initialized: now 1786115229955342 us; error 33 us; skew 500 ppm
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:09.957906 30716 init.cc:377] Logged in from keytab as oryx/127.18.129.3@KRBTEST.COM (short username oryx)
I20260807 15:07:09.958365 30716 webserver.cc:460] Webserver started at http://127.18.129.3:42061/ using document root <none> and password file <none>
I20260807 15:07:09.958575 30716 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:09.958623 30716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:09.958746 30716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:09.959939 30716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "b0e4c329706743efb1ba042c669a7d98"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "dfa8045d235b267c4045f85ca3463924"
server_key_iv: "13c31b18d7c3b870c41daea66bd648f7"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.960304 30716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "b0e4c329706743efb1ba042c669a7d98"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "dfa8045d235b267c4045f85ca3463924"
server_key_iv: "13c31b18d7c3b870c41daea66bd648f7"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.961553 30716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:07:09.962245 30744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:09.962401 30716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:07:09.962487 30716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "b0e4c329706743efb1ba042c669a7d98"
format_stamp: "Formatted at 2026-08-07 15:07:09 on dist-test-slave-5bn2"
server_key: "dfa8045d235b267c4045f85ca3463924"
server_key_iv: "13c31b18d7c3b870c41daea66bd648f7"
server_key_version: "encryptionkey@0"
I20260807 15:07:09.962554 30716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:09.974164 30716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:09.976159 30716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:09.976315 30716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:09.976913 30716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:09.976960 30716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:09.976994 30716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:09.977017 30716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:07:09 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:09.984746 30716 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:42485
I20260807 15:07:09.984800 30937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:42485 every 8 connection(s)
I20260807 15:07:09.985183 30716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:07:09.986037 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 30716
I20260807 15:07:09.986169 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:07:09.986347 18948 external_mini_cluster.cc:1428] Setting key f5822e7709710c566a6fd276896c130e
I20260807 15:07:09.986634 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.977251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:35177 (local address 127.18.129.62:35845)
0807 15:07:09.977322 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:09.977325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:09.977530 (+ 205us) server_negotiation.cc:407] Connection header received
0807 15:07:09.978020 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:09.978022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:09.978036 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:09.978128 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:09.978735 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.979293 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.980294 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.980592 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.980687 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.983437 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:09.983438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:09.983439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:09.983446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:09.985054 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.985470 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.985472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.985472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.985555 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.985868 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.985869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.985869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.986026 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:09.986105 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:09.986436 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 15:07:09.986533 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:07:09.986776 30759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.977242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35845 (local address 127.18.129.3:35177)
0807 15:07:09.977398 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:09.977437 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:09.977924 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:09.978175 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:09.978178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:09.978408 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:09.978649 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.978655 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.979376 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.979377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.980213 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.980216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.980697 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.980698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.980836 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.981401 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:09.981411 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:09.983348 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:09.985135 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.985138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.985140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.985386 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.985625 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.985626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.985626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.985771 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.986097 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:09.986105 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:09.986266 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 15:07:09.986382 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 15:07:09.986554 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260807 15:07:09.987298 30938 heartbeater.cc:347] Connected to a master server at 127.18.129.62:35845
I20260807 15:07:09.987378 30938 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:09.987577 30938 heartbeater.cc:511] Master 127.18.129.62:35845 requested a full tablet report, sending...
I20260807 15:07:09.988412 30143 ts_manager.cc:194] Registered new tserver with Master: b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485)
I20260807 15:07:09.988806 30143 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:35177
I20260807 15:07:09.993561 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.988362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:46001 (local address 127.18.129.62:35845)
0807 15:07:09.988396 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:09.988398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:09.989165 (+ 767us) server_negotiation.cc:407] Connection header received
0807 15:07:09.989226 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:09.989226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:09.989235 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:09.989277 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:09.989618 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.990072 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.990685 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.990879 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.990939 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.991696 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:09.991696 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:09.991696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:09.991701 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:09.992719 (+ 1018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.992984 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.992985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.992985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.993057 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:09.993193 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:09.993194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:09.993194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:09.993305 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:09.993356 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:09.993445 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 15:07:09.993502 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260807 15:07:09.993562 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:09.988975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35845 (local address 127.18.129.3:46001)
0807 15:07:09.989101 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:09.989112 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:09.989173 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:09.989282 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:09.989283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:09.989485 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:09.989542 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.989544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.990145 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.990146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.990631 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:09.990633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:09.990941 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:09.990941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:09.990985 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:09.991357 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:09.991362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:09.991636 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:09.992780 (+ 1144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.992781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.992781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.992932 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.993113 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:09.993113 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:09.993113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:09.993156 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:09.993351 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:09.993352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:09.993385 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 15:07:09.993428 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:07:09.993498 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260807 15:07:09.997991 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115230, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:07:10.020131 30234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:10.014883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.18.129.62:35845)
0807 15:07:10.014944 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.014946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.014958 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:07:10.015000 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.015001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.015010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.015051 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:10.015607 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.016105 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.016832 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.017019 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.017076 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.020019 (+ 2943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59366: BlockingRecv error: recv got EOF from 127.0.0.1:59366 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115230, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:10.028012 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.020763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.18.129.62:35845)
0807 15:07:10.020825 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.020827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.020868 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:07:10.020919 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.020920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.020928 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.020971 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:10.021360 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.021827 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.022428 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.022598 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.022640 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.024795 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:10.024797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:10.024797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:10.024806 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:10.026445 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:10.026819 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:10.026821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:10.026821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:10.027168 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:10.027463 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:10.027464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:10.027464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:10.027624 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:10.027730 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.027831 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.027915 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:07:10.032541 30143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:07:10.034827 30143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:10.041368 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.037134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43023 (local address 127.0.0.1:52292)
0807 15:07:10.037618 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.037627 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.037674 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.037868 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.037872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.037875 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.037942 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.037944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.039265 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.039266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.040424 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.040426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.041075 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.041076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.041155 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.041170 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.041229 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.041284 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":32,"thread_start_us":91,"threads_started":1}
I20260807 15:07:10.041390 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.037323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.18.129.2:43023)
0807 15:07:10.037535 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.037541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.037686 (+ 145us) server_negotiation.cc:407] Connection header received
0807 15:07:10.037735 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.037738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.037755 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.037832 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.038023 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.039176 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.040483 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.040992 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.041062 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.041153 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.041254 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.041295 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":104,"threads_started":1}
I20260807 15:07:10.041487 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.037132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:42485 (local address 127.0.0.1:41676)
0807 15:07:10.037493 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.037501 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.037536 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.037967 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.037968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.037971 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.038038 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.038040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.039301 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.039302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.040228 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.040231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.041233 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.041235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.041314 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.041319 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.041363 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.041404 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":94,"threads_started":1}
I20260807 15:07:10.041646 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.037130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35383 (local address 127.0.0.1:34810)
0807 15:07:10.037291 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.037333 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.037398 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.038150 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.038150 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.038153 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.038200 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.038201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.039100 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.039101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.040353 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.040356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.041348 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.041349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.041416 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.041419 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.041461 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.041492 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":45,"threads_started":1}
I20260807 15:07:10.041683 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.037378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.18.129.1:35383)
0807 15:07:10.037930 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.037934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.037944 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:07:10.037980 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.037982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.037996 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.038083 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.038269 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.039025 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.040443 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.041183 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.041244 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.041318 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.041538 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.041587 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":74,"threads_started":1}
I20260807 15:07:10.041939 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.037333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.18.129.3:42485)
0807 15:07:10.037624 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.037628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.037640 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:07:10.037681 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.037683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.037722 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.037796 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.038102 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.039203 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.040303 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.041053 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.041124 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.041196 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.041805 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.041847 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":112,"threads_started":1}
I20260807 15:07:10.042290 30590 tablet_service.cc:1467] Processing CreateTablet for tablet a1372280e68a4b3fa20a764f67babb68 (DEFAULT_TABLE table=test-table [id=e7d561aeafa9401e9b345e33950585a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.042290 30375 tablet_service.cc:1467] Processing CreateTablet for tablet a1372280e68a4b3fa20a764f67babb68 (DEFAULT_TABLE table=test-table [id=e7d561aeafa9401e9b345e33950585a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.042659 30375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1372280e68a4b3fa20a764f67babb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.042659 30590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1372280e68a4b3fa20a764f67babb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.046581 30979 tablet_bootstrap.cc:492] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Bootstrap starting.
I20260807 15:07:10.046973 30980 tablet_bootstrap.cc:492] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Bootstrap starting.
I20260807 15:07:10.046964 30826 tablet_service.cc:1467] Processing CreateTablet for tablet a1372280e68a4b3fa20a764f67babb68 (DEFAULT_TABLE table=test-table [id=e7d561aeafa9401e9b345e33950585a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.047288 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1372280e68a4b3fa20a764f67babb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.047348 30979 tablet_bootstrap.cc:654] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.047600 30980 tablet_bootstrap.cc:654] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.047699 30979 log.cc:824] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:10.047916 30980 log.cc:824] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:10.048552 30979 tablet_bootstrap.cc:492] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: No bootstrap required, opened a new log
I20260807 15:07:10.048619 30979 ts_tablet_manager.cc:1397] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:10.048740 30980 tablet_bootstrap.cc:492] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: No bootstrap required, opened a new log
I20260807 15:07:10.048818 30980 ts_tablet_manager.cc:1397] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:10.050186 30979 raft_consensus.cc:348] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.050320 30979 raft_consensus.cc:374] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.050354 30979 raft_consensus.cc:729] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51cc4513040492d816536dc9db09fed, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.050464 30979 consensus_queue.cc:260] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.050734 30979 ts_tablet_manager.cc:1428] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:07:10.050895 30474 heartbeater.cc:503] Master 127.18.129.62:35845 was elected leader, sending a full tablet report...
I20260807 15:07:10.050964 30980 raft_consensus.cc:348] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.051095 30985 tablet_bootstrap.cc:492] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: Bootstrap starting.
I20260807 15:07:10.051138 30980 raft_consensus.cc:374] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.051180 30980 raft_consensus.cc:729] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee6853fb1f44f21afaa25e67072871c, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.051312 30980 consensus_queue.cc:260] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.051584 30980 ts_tablet_manager.cc:1428] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:07:10.051708 30985 tablet_bootstrap.cc:654] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.051837 30701 heartbeater.cc:503] Master 127.18.129.62:35845 was elected leader, sending a full tablet report...
I20260807 15:07:10.052014 30985 log.cc:824] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:10.052747 30985 tablet_bootstrap.cc:492] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: No bootstrap required, opened a new log
I20260807 15:07:10.052817 30985 ts_tablet_manager.cc:1397] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:10.054111 30985 raft_consensus.cc:348] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.054240 30985 raft_consensus.cc:374] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.054284 30985 raft_consensus.cc:729] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e4c329706743efb1ba042c669a7d98, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.055059 30985 consensus_queue.cc:260] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.055330 30985 ts_tablet_manager.cc:1428] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:07:10.055397 30938 heartbeater.cc:503] Master 127.18.129.62:35845 was elected leader, sending a full tablet report...
W20260807 15:07:10.073378 30702 tablet.cc:2369] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:07:10.157836 30984 raft_consensus.cc:482] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:10.157958 30984 raft_consensus.cc:504] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.158335 30984 leader_election.cc:288] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485), 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023)
I20260807 15:07:10.161151 30304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.158503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:42485 (local address 127.18.129.1:33401)
0807 15:07:10.158619 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.158631 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.158676 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.158817 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.158818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.158820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.158917 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.158919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.159610 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.159610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.160344 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.160346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.160874 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.160875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.160961 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.160964 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.161009 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.161049 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":2}
I20260807 15:07:10.161137 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.158599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:60877 (local address 127.18.129.2:43023)
0807 15:07:10.158645 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.158647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.158730 (+ 83us) server_negotiation.cc:407] Connection header received
0807 15:07:10.158768 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.158769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.158779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.158832 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.159015 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.159471 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.160437 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.160802 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.160864 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.160871 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.161030 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.161063 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:07:10.161139 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.158631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:33401 (local address 127.18.129.3:42485)
0807 15:07:10.158693 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.158695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.158718 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:07:10.158746 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.158747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.158756 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.158800 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.158980 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.159529 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.160444 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.160800 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.160862 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.160871 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.161030 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.161073 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:07:10.161170 30995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.158516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43023 (local address 127.18.129.1:60877)
0807 15:07:10.158651 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.158660 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.158725 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.158865 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.158866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.158868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.158946 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.158948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.159547 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.159548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.160348 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.160351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.160877 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.160878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.160965 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.160968 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.161017 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.161061 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260807 15:07:10.161491 30625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1372280e68a4b3fa20a764f67babb68" candidate_uuid: "f51cc4513040492d816536dc9db09fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ee6853fb1f44f21afaa25e67072871c" is_pre_election: true
I20260807 15:07:10.161491 30858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1372280e68a4b3fa20a764f67babb68" candidate_uuid: "f51cc4513040492d816536dc9db09fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0e4c329706743efb1ba042c669a7d98" is_pre_election: true
I20260807 15:07:10.161651 30625 raft_consensus.cc:2454] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51cc4513040492d816536dc9db09fed in term 0.
I20260807 15:07:10.161651 30858 raft_consensus.cc:2454] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51cc4513040492d816536dc9db09fed in term 0.
I20260807 15:07:10.161875 30296 leader_election.cc:302] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ee6853fb1f44f21afaa25e67072871c, f51cc4513040492d816536dc9db09fed; no voters:
I20260807 15:07:10.162005 30984 raft_consensus.cc:2781] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:10.162070 30984 raft_consensus.cc:482] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:10.162091 30984 raft_consensus.cc:3037] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.163053 30984 raft_consensus.cc:504] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.163174 30984 leader_election.cc:288] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [CANDIDATE]: Term 1 election: Requested vote from peers b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485), 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023)
I20260807 15:07:10.163326 30625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1372280e68a4b3fa20a764f67babb68" candidate_uuid: "f51cc4513040492d816536dc9db09fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ee6853fb1f44f21afaa25e67072871c"
I20260807 15:07:10.163398 30858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1372280e68a4b3fa20a764f67babb68" candidate_uuid: "f51cc4513040492d816536dc9db09fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0e4c329706743efb1ba042c669a7d98"
I20260807 15:07:10.163422 30625 raft_consensus.cc:3037] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.163480 30858 raft_consensus.cc:3037] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.164326 30625 raft_consensus.cc:2454] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51cc4513040492d816536dc9db09fed in term 1.
I20260807 15:07:10.164326 30858 raft_consensus.cc:2454] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51cc4513040492d816536dc9db09fed in term 1.
I20260807 15:07:10.164517 30292 leader_election.cc:302] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0e4c329706743efb1ba042c669a7d98, f51cc4513040492d816536dc9db09fed; no voters:
I20260807 15:07:10.164623 30984 raft_consensus.cc:2781] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:10.164793 30984 raft_consensus.cc:686] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 1 LEADER]: Becoming Leader. State: Replica: f51cc4513040492d816536dc9db09fed, State: Running, Role: LEADER
I20260807 15:07:10.164881 30984 consensus_queue.cc:237] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } }
I20260807 15:07:10.165789 30143 catalog_manager.cc:5495] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed reported cstate change: term changed from 0 to 1, leader changed from <none> to f51cc4513040492d816536dc9db09fed (127.18.129.1). New cstate: current_term: 1 leader_uuid: "f51cc4513040492d816536dc9db09fed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } health_report { overall_health: UNKNOWN } } }
W20260807 15:07:10.178532 30475 tablet.cc:2369] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:07:10.190303 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.187973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.18.129.1:35383)
0807 15:07:10.188040 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.188042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.188051 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:07:10.188092 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.188093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.188114 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.188184 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:10.188330 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.188809 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.189550 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.189765 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.189820 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.189973 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.190079 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.190166 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.190204 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.190225 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:07:10.192071 30625 raft_consensus.cc:1261] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Refusing update from remote peer f51cc4513040492d816536dc9db09fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.192071 30858 raft_consensus.cc:1261] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Refusing update from remote peer f51cc4513040492d816536dc9db09fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.192435 30996 consensus_queue.cc:1035] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.192584 30984 consensus_queue.cc:1035] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.194824 31002 mvcc.cc:205] Tried to move back new op lower bound from 7315927982863884288 to 7315927982755938304. Current Snapshot: MvccSnapshot[applied={T|T < 7315927982863884288}]
I20260807 15:07:10.195490 31004 mvcc.cc:205] Tried to move back new op lower bound from 7315927982863884288 to 7315927982755938304. Current Snapshot: MvccSnapshot[applied={T|T < 7315927982863884288}]
I20260807 15:07:10.197168 30143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:10.197252 30143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:10.197966 30143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: Sending DeleteTablet for 3 replicas of tablet a1372280e68a4b3fa20a764f67babb68
I20260807 15:07:10.198293 30826 tablet_service.cc:1514] Processing DeleteTablet for tablet a1372280e68a4b3fa20a764f67babb68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41676
I20260807 15:07:10.198431 30590 tablet_service.cc:1514] Processing DeleteTablet for tablet a1372280e68a4b3fa20a764f67babb68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:52292
I20260807 15:07:10.198698 31019 tablet_replica.cc:330] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: stopping tablet replica
I20260807 15:07:10.198786 31018 tablet_replica.cc:330] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: stopping tablet replica
I20260807 15:07:10.198801 31019 raft_consensus.cc:2229] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:10.198920 31018 raft_consensus.cc:2229] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:10.198933 31019 raft_consensus.cc:2258] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.199014 30143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:07:10.199126 31018 raft_consensus.cc:2258] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 15:07:10.199169 30143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:10.199348 31019 ts_tablet_manager.cc:1905] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.199404 30375 tablet_service.cc:1514] Processing DeleteTablet for tablet a1372280e68a4b3fa20a764f67babb68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:34810
I20260807 15:07:10.199529 31018 ts_tablet_manager.cc:1905] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.201068 31021 tablet_replica.cc:330] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: stopping tablet replica
I20260807 15:07:10.201211 30590 tablet_service.cc:1467] Processing CreateTablet for tablet ac5921834108478eb9d2066292cbc834 (DEFAULT_TABLE table=test-table [id=a16b890c3638472c84073fad12d07268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.201316 30590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac5921834108478eb9d2066292cbc834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.201328 30826 tablet_service.cc:1467] Processing CreateTablet for tablet ac5921834108478eb9d2066292cbc834 (DEFAULT_TABLE table=test-table [id=a16b890c3638472c84073fad12d07268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.201412 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac5921834108478eb9d2066292cbc834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.201572 31018 ts_tablet_manager.cc:1918] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:10.201642 31018 log.cc:1196] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/a1372280e68a4b3fa20a764f67babb68
I20260807 15:07:10.201915 31018 ts_tablet_manager.cc:1939] T a1372280e68a4b3fa20a764f67babb68 P b0e4c329706743efb1ba042c669a7d98: Deleting consensus metadata
I20260807 15:07:10.202450 30980 tablet_bootstrap.cc:492] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: Bootstrap starting.
I20260807 15:07:10.202600 30985 tablet_bootstrap.cc:492] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: Bootstrap starting.
I20260807 15:07:10.202693 30126 catalog_manager.cc:4841] TS b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485): tablet a1372280e68a4b3fa20a764f67babb68 (table test-table [id=e7d561aeafa9401e9b345e33950585a0]) successfully deleted
I20260807 15:07:10.202894 30980 tablet_bootstrap.cc:654] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.202991 30375 tablet_service.cc:1467] Processing CreateTablet for tablet ac5921834108478eb9d2066292cbc834 (DEFAULT_TABLE table=test-table [id=a16b890c3638472c84073fad12d07268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.203030 30985 tablet_bootstrap.cc:654] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.203109 30375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac5921834108478eb9d2066292cbc834. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.201205 31019 ts_tablet_manager.cc:1918] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:10.203243 31019 log.cc:1196] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/a1372280e68a4b3fa20a764f67babb68
I20260807 15:07:10.203465 31019 ts_tablet_manager.cc:1939] T a1372280e68a4b3fa20a764f67babb68 P 7ee6853fb1f44f21afaa25e67072871c: Deleting consensus metadata
I20260807 15:07:10.203611 30980 tablet_bootstrap.cc:492] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: No bootstrap required, opened a new log
I20260807 15:07:10.203657 30980 ts_tablet_manager.cc:1397] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:10.203823 31021 raft_consensus.cc:2229] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:07:10.203794 30980 raft_consensus.cc:348] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.203871 30980 raft_consensus.cc:374] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.203893 30980 raft_consensus.cc:729] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee6853fb1f44f21afaa25e67072871c, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.203928 31021 raft_consensus.cc:2258] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.203934 30980 consensus_queue.cc:260] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.204015 30980 ts_tablet_manager.cc:1428] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:10.204253 30130 catalog_manager.cc:4841] TS 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023): tablet a1372280e68a4b3fa20a764f67babb68 (table test-table [id=e7d561aeafa9401e9b345e33950585a0]) successfully deleted
I20260807 15:07:10.204353 31021 ts_tablet_manager.cc:1905] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.204420 30979 tablet_bootstrap.cc:492] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: Bootstrap starting.
I20260807 15:07:10.204793 30979 tablet_bootstrap.cc:654] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.205416 30979 tablet_bootstrap.cc:492] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: No bootstrap required, opened a new log
I20260807 15:07:10.205467 30979 ts_tablet_manager.cc:1397] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:10.205590 30979 raft_consensus.cc:348] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.205651 30979 raft_consensus.cc:374] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.205670 30979 raft_consensus.cc:729] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51cc4513040492d816536dc9db09fed, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.205716 30979 consensus_queue.cc:260] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.205763 31021 ts_tablet_manager.cc:1918] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:10.205792 30979 ts_tablet_manager.cc:1428] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.205832 31021 log.cc:1196] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/a1372280e68a4b3fa20a764f67babb68
I20260807 15:07:10.206065 31021 ts_tablet_manager.cc:1939] T a1372280e68a4b3fa20a764f67babb68 P f51cc4513040492d816536dc9db09fed: Deleting consensus metadata
I20260807 15:07:10.206424 30127 catalog_manager.cc:4841] TS f51cc4513040492d816536dc9db09fed (127.18.129.1:35383): tablet a1372280e68a4b3fa20a764f67babb68 (table test-table [id=e7d561aeafa9401e9b345e33950585a0]) successfully deleted
I20260807 15:07:10.206605 30985 tablet_bootstrap.cc:492] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: No bootstrap required, opened a new log
I20260807 15:07:10.206650 30985 ts_tablet_manager.cc:1397] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 15:07:10.206784 30985 raft_consensus.cc:348] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.206842 30985 raft_consensus.cc:374] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.206897 30985 raft_consensus.cc:729] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e4c329706743efb1ba042c669a7d98, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.206941 30985 consensus_queue.cc:260] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.207032 30985 ts_tablet_manager.cc:1428] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
W20260807 15:07:10.235831 30939 tablet.cc:2369] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:07:10.246093 30987 raft_consensus.cc:482] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:10.246192 30987 raft_consensus.cc:504] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.246505 30987 leader_election.cc:288] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f51cc4513040492d816536dc9db09fed (127.18.129.1:35383), b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485)
I20260807 15:07:10.249509 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.246793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:58965 (local address 127.18.129.3:42485)
0807 15:07:10.246840 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.246842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.247105 (+ 263us) server_negotiation.cc:407] Connection header received
0807 15:07:10.247141 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.247142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.247151 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.247197 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.247337 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.247825 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.248769 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.249153 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.249217 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.249226 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.249395 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.249435 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:07:10.249578 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.246677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:42485 (local address 127.18.129.2:58965)
0807 15:07:10.247042 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.247050 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.247097 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.247198 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.247199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.247200 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.247282 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.247283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.247900 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.247901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.248687 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.248690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.249266 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.249267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.249333 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.249338 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.249385 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.249421 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":191,"thread_start_us":114,"threads_started":1}
I20260807 15:07:10.249830 30858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac5921834108478eb9d2066292cbc834" candidate_uuid: "7ee6853fb1f44f21afaa25e67072871c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0e4c329706743efb1ba042c669a7d98" is_pre_election: true
I20260807 15:07:10.249930 30858 raft_consensus.cc:2454] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee6853fb1f44f21afaa25e67072871c in term 0.
I20260807 15:07:10.250025 30532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.246677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35383 (local address 127.18.129.2:48727)
0807 15:07:10.246745 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.246757 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.246799 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.246976 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.246977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.246979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.247059 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.247061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.247998 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.247999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.249111 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.249114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.249773 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.249774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.249853 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.249856 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.249920 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.249961 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 15:07:10.250113 30518 leader_election.cc:302] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ee6853fb1f44f21afaa25e67072871c, b0e4c329706743efb1ba042c669a7d98; no voters:
I20260807 15:07:10.250247 30987 raft_consensus.cc:2781] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:10.250289 30987 raft_consensus.cc:482] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:10.250308 30987 raft_consensus.cc:3037] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.250317 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.246808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:48727 (local address 127.18.129.1:35383)
0807 15:07:10.246867 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.246869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.246876 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:07:10.246903 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.246904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.246913 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.246965 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.247154 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.247900 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.249199 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.249664 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.249736 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.249744 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.250212 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.250254 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:07:10.250483 30402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac5921834108478eb9d2066292cbc834" candidate_uuid: "7ee6853fb1f44f21afaa25e67072871c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51cc4513040492d816536dc9db09fed" is_pre_election: true
I20260807 15:07:10.250623 30402 raft_consensus.cc:2454] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee6853fb1f44f21afaa25e67072871c in term 0.
I20260807 15:07:10.251238 30987 raft_consensus.cc:504] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.251382 30987 leader_election.cc:288] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [CANDIDATE]: Term 1 election: Requested vote from peers f51cc4513040492d816536dc9db09fed (127.18.129.1:35383), b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485)
I20260807 15:07:10.251500 30402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac5921834108478eb9d2066292cbc834" candidate_uuid: "7ee6853fb1f44f21afaa25e67072871c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51cc4513040492d816536dc9db09fed"
I20260807 15:07:10.251564 30402 raft_consensus.cc:3037] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.251587 30858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac5921834108478eb9d2066292cbc834" candidate_uuid: "7ee6853fb1f44f21afaa25e67072871c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0e4c329706743efb1ba042c669a7d98"
I20260807 15:07:10.251668 30858 raft_consensus.cc:3037] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.252151 30402 raft_consensus.cc:2454] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee6853fb1f44f21afaa25e67072871c in term 1.
I20260807 15:07:10.252342 30519 leader_election.cc:302] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ee6853fb1f44f21afaa25e67072871c, f51cc4513040492d816536dc9db09fed; no voters:
I20260807 15:07:10.252420 30987 raft_consensus.cc:2781] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:10.252434 30858 raft_consensus.cc:2454] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee6853fb1f44f21afaa25e67072871c in term 1.
I20260807 15:07:10.252575 30987 raft_consensus.cc:686] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 1 LEADER]: Becoming Leader. State: Replica: 7ee6853fb1f44f21afaa25e67072871c, State: Running, Role: LEADER
I20260807 15:07:10.252651 30987 consensus_queue.cc:237] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.253255 30143 catalog_manager.cc:5495] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2). New cstate: current_term: 1 leader_uuid: "7ee6853fb1f44f21afaa25e67072871c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115229, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.62@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:10.287487 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.279629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.18.129.62:35845)
0807 15:07:10.279678 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.279680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.279806 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:07:10.279937 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.279938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.279948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.279999 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:10.280608 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.281147 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.282136 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.282321 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.282372 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.284594 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:10.284596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:10.284596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:10.284602 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:10.285869 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:10.286323 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:10.286325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:10.286325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:10.286460 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:10.286706 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:10.286706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:10.286707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:10.286892 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:10.286974 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.287144 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.287314 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":26}
I20260807 15:07:10.287487 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.279531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35845 (local address 127.0.0.1:59388)
0807 15:07:10.279718 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.279731 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.279868 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.280022 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.280023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.280468 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:10.280533 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.280536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.281223 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.281224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.282052 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.282055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.282389 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.282389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.282447 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.283014 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:10.283019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:10.284504 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:10.285942 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:10.285949 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:10.285951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:10.286181 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:10.286578 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:10.286579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:10.286579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:10.286647 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:10.286969 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:10.286978 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:10.287032 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.287111 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.287245 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260807 15:07:10.289665 30143 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:59388:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:07:10.289921 30143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:10.291527 30375 tablet_service.cc:1467] Processing CreateTablet for tablet 267f1d3cb7794191b0e8e99aade6b708 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acfefcb5551143ddb22957403a0f8351]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:07:10.291613 30826 tablet_service.cc:1467] Processing CreateTablet for tablet 267f1d3cb7794191b0e8e99aade6b708 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acfefcb5551143ddb22957403a0f8351]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:07:10.291613 30590 tablet_service.cc:1467] Processing CreateTablet for tablet 267f1d3cb7794191b0e8e99aade6b708 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acfefcb5551143ddb22957403a0f8351]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:07:10.291692 30375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267f1d3cb7794191b0e8e99aade6b708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.291779 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267f1d3cb7794191b0e8e99aade6b708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.291777 30590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 267f1d3cb7794191b0e8e99aade6b708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.293251 30979 tablet_bootstrap.cc:492] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: Bootstrap starting.
I20260807 15:07:10.293336 30980 tablet_bootstrap.cc:492] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: Bootstrap starting.
I20260807 15:07:10.293691 30979 tablet_bootstrap.cc:654] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.293702 30980 tablet_bootstrap.cc:654] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.294031 30985 tablet_bootstrap.cc:492] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: Bootstrap starting.
I20260807 15:07:10.294453 30985 tablet_bootstrap.cc:654] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.294531 30980 tablet_bootstrap.cc:492] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: No bootstrap required, opened a new log
I20260807 15:07:10.294579 30980 ts_tablet_manager.cc:1397] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:10.294912 30980 raft_consensus.cc:348] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.295008 30980 raft_consensus.cc:374] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.295032 30980 raft_consensus.cc:729] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee6853fb1f44f21afaa25e67072871c, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.295078 30980 consensus_queue.cc:260] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.295142 30985 tablet_bootstrap.cc:492] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: No bootstrap required, opened a new log
I20260807 15:07:10.295177 30980 ts_tablet_manager.cc:1428] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:10.295186 30985 ts_tablet_manager.cc:1397] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:07:10.295296 30987 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.295362 30987 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:07:10.295360 30985 raft_consensus.cc:348] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.295430 30985 raft_consensus.cc:374] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.295451 30985 raft_consensus.cc:729] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e4c329706743efb1ba042c669a7d98, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.295469 30979 tablet_bootstrap.cc:492] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: No bootstrap required, opened a new log
I20260807 15:07:10.295512 30979 ts_tablet_manager.cc:1397] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:10.295495 30985 consensus_queue.cc:260] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.295588 30985 ts_tablet_manager.cc:1428] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.295653 30979 raft_consensus.cc:348] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.295714 30979 raft_consensus.cc:374] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.295732 30979 raft_consensus.cc:729] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51cc4513040492d816536dc9db09fed, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.295773 30979 consensus_queue.cc:260] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.295863 30979 ts_tablet_manager.cc:1428] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.296008 30990 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.296100 30990 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:07:10.296281 30996 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.296357 30996 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:07:10.415356 30990 raft_consensus.cc:482] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:10.415467 30990 raft_consensus.cc:504] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.415818 30990 leader_election.cc:288] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f51cc4513040492d816536dc9db09fed (127.18.129.1:35383), 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023)
I20260807 15:07:10.418814 30759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.415993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35383 (local address 127.18.129.3:55953)
0807 15:07:10.416054 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.416066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.416106 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.416245 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.416246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.416248 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.416321 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.416324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.416992 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.416993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.417872 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.417875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.418621 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.418622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.418689 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.418692 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.418729 (+ 37us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.418759 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:07:10.418879 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.416076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:55953 (local address 127.18.129.1:35383)
0807 15:07:10.416123 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.416125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.416148 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:07:10.416176 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.416177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.416186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.416226 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.416395 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.416918 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.417966 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.418519 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.418616 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.418626 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.418747 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.418784 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:07:10.419093 30402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267f1d3cb7794191b0e8e99aade6b708" candidate_uuid: "b0e4c329706743efb1ba042c669a7d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51cc4513040492d816536dc9db09fed" is_pre_election: true
I20260807 15:07:10.419181 30402 raft_consensus.cc:2454] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0e4c329706743efb1ba042c669a7d98 in term 0.
I20260807 15:07:10.419348 30748 leader_election.cc:302] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0e4c329706743efb1ba042c669a7d98, f51cc4513040492d816536dc9db09fed; no voters:
I20260807 15:07:10.419502 30990 raft_consensus.cc:2781] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:10.419538 30990 raft_consensus.cc:482] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:10.419559 30990 raft_consensus.cc:3037] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.420111 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.416105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:48615 (local address 127.18.129.2:43023)
0807 15:07:10.416146 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.416148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.416400 (+ 252us) server_negotiation.cc:407] Connection header received
0807 15:07:10.416501 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.416502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.416511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.416555 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:10.416699 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.417287 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.419329 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.419810 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.419866 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.419872 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.420021 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.420047 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:07:10.420142 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.415994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43023 (local address 127.18.129.3:48615)
0807 15:07:10.416346 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.416354 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.416435 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.416566 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.416567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.416568 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:10.416647 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.416649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.417376 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.417377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.419228 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.419232 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.419884 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.419885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.419954 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.419958 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.420017 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.420052 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":154,"thread_start_us":116,"threads_started":1}
I20260807 15:07:10.420418 30990 raft_consensus.cc:504] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.420487 30625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267f1d3cb7794191b0e8e99aade6b708" candidate_uuid: "b0e4c329706743efb1ba042c669a7d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ee6853fb1f44f21afaa25e67072871c" is_pre_election: true
I20260807 15:07:10.420554 30990 leader_election.cc:288] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [CANDIDATE]: Term 1 election: Requested vote from peers f51cc4513040492d816536dc9db09fed (127.18.129.1:35383), 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023)
I20260807 15:07:10.420584 30625 raft_consensus.cc:2454] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0e4c329706743efb1ba042c669a7d98 in term 0.
I20260807 15:07:10.420711 30625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267f1d3cb7794191b0e8e99aade6b708" candidate_uuid: "b0e4c329706743efb1ba042c669a7d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ee6853fb1f44f21afaa25e67072871c"
I20260807 15:07:10.420707 30402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "267f1d3cb7794191b0e8e99aade6b708" candidate_uuid: "b0e4c329706743efb1ba042c669a7d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51cc4513040492d816536dc9db09fed"
I20260807 15:07:10.420773 30625 raft_consensus.cc:3037] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.420775 30402 raft_consensus.cc:3037] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.421496 30402 raft_consensus.cc:2454] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0e4c329706743efb1ba042c669a7d98 in term 1.
I20260807 15:07:10.421496 30625 raft_consensus.cc:2454] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0e4c329706743efb1ba042c669a7d98 in term 1.
I20260807 15:07:10.421671 30748 leader_election.cc:302] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0e4c329706743efb1ba042c669a7d98, f51cc4513040492d816536dc9db09fed; no voters:
I20260807 15:07:10.421769 30990 raft_consensus.cc:2781] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:10.421939 30990 raft_consensus.cc:686] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [term 1 LEADER]: Becoming Leader. State: Replica: b0e4c329706743efb1ba042c669a7d98, State: Running, Role: LEADER
I20260807 15:07:10.422001 30990 consensus_queue.cc:237] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.422325 31050 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: TxnStatusTablet state changed. Reason: New leader b0e4c329706743efb1ba042c669a7d98. Latest consensus state: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.422411 31050 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:07:10.422636 31054 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:07:10.422739 31054 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:07:10.422667 30143 catalog_manager.cc:5495] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0e4c329706743efb1ba042c669a7d98 (127.18.129.3). New cstate: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } health_report { overall_health: HEALTHY } } }
I20260807 15:07:10.441985 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.439317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:42485 (local address 127.0.0.1:41678)
0807 15:07:10.439389 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.439401 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.439468 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.439624 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.439624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.439625 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:07:10.439673 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.439675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.440520 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.440520 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.441073 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.441074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.441520 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.441521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.441576 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.441591 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.441850 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.441854 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.441877 (+ 23us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.441916 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 15:07:10.441982 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.439374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41678 (local address 127.18.129.3:42485)
0807 15:07:10.439424 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.439426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.439453 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:07:10.439530 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.439531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.439551 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.439622 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:10.439713 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.440457 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.441137 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.441435 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.441502 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.441673 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.441795 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.441864 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.441881 (+ 17us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.441900 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:07:10.443086 30625 raft_consensus.cc:1261] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Refusing update from remote peer b0e4c329706743efb1ba042c669a7d98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.443195 30402 raft_consensus.cc:1261] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Refusing update from remote peer b0e4c329706743efb1ba042c669a7d98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.443318 31050 consensus_queue.cc:1035] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.443431 30990 consensus_queue.cc:1035] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.443986 30987 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: TxnStatusTablet state changed. Reason: New leader b0e4c329706743efb1ba042c669a7d98. Latest consensus state: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.444073 30987 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:07:10.444033 30996 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: TxnStatusTablet state changed. Reason: New leader b0e4c329706743efb1ba042c669a7d98. Latest consensus state: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.444128 30996 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:07:10.444494 30990 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.444573 30990 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:07:10.444861 31006 mvcc.cc:205] Tried to move back new op lower bound from 7315927983892799488 to 7315927983808995328. Current Snapshot: MvccSnapshot[applied={T|T < 7315927983892799488 or (T in {7315927983892799488})}]
I20260807 15:07:10.444989 30987 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.445071 30987 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P 7ee6853fb1f44f21afaa25e67072871c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:07:10.447084 31050 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.447160 31050 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P b0e4c329706743efb1ba042c669a7d98: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:07:10.447582 30996 tablet_replica.cc:439] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } } }
I20260807 15:07:10.447669 30996 tablet_replica.cc:442] T 267f1d3cb7794191b0e8e99aade6b708 P f51cc4513040492d816536dc9db09fed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:07:10.448791 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.446171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.18.129.2:43023)
0807 15:07:10.446296 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.446298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.446304 (+ 6us) server_negotiation.cc:407] Connection header received
0807 15:07:10.446338 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.446338 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.446359 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.446417 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:10.446571 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.447163 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.448116 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.448310 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.448355 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.448526 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.448612 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.448660 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.448715 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.448732 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 15:07:10.457022 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.451033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:42877 (local address 127.18.129.3:42485)
0807 15:07:10.451086 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.451088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.455053 (+ 3965us) server_negotiation.cc:407] Connection header received
0807 15:07:10.455100 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.455100 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.455110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.455157 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:10.455269 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.455716 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.456412 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.456600 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.456653 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.456805 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.456835 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.456874 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.456923 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.456949 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:07:10.457042 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.450973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:42485 (local address 127.18.129.2:42877)
0807 15:07:10.454983 (+ 4010us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.454994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.455050 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.455163 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.455163 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.455164 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:07:10.455217 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.455218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.455784 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.455785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.456342 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.456344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.456660 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.456661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.456726 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.456740 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.456880 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.456882 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.456922 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.456962 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3971,"thread_start_us":82,"threads_started":1}
I20260807 15:07:10.459025 30858 raft_consensus.cc:1261] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Refusing update from remote peer 7ee6853fb1f44f21afaa25e67072871c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.459160 30402 raft_consensus.cc:1261] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Refusing update from remote peer 7ee6853fb1f44f21afaa25e67072871c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.459312 30987 consensus_queue.cc:1035] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.459450 31030 consensus_queue.cc:1035] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.467494 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.465109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:51991 (local address 127.18.129.2:43023)
0807 15:07:10.465144 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.465146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.465181 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:07:10.465252 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.465252 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.465261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.465303 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:10.465459 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.465962 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.466783 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.467066 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.467121 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.467279 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.467315 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.467360 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.467419 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.467444 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:07:10.467530 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.465072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43023 (local address 127.18.129.3:51991)
0807 15:07:10.465131 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.465143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:10.465197 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:10.465327 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:10.465328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:10.465330 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:07:10.465388 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.465390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.466022 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.466023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.466714 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:10.466716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.467123 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.467124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:10.467196 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.467212 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.467364 (+ 152us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.467366 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:07:10.467408 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:07:10.467454 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 15:07:10.474157 30144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:10.474254 30144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:10.474826 30144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: Sending DeleteTablet for 3 replicas of tablet ac5921834108478eb9d2066292cbc834
I20260807 15:07:10.475107 30590 tablet_service.cc:1514] Processing DeleteTablet for tablet ac5921834108478eb9d2066292cbc834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:52292
I20260807 15:07:10.475203 30375 tablet_service.cc:1514] Processing DeleteTablet for tablet ac5921834108478eb9d2066292cbc834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:34810
I20260807 15:07:10.475210 31019 tablet_replica.cc:330] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: stopping tablet replica
I20260807 15:07:10.475283 31019 raft_consensus.cc:2229] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:07:10.475292 31021 tablet_replica.cc:330] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: stopping tablet replica
I20260807 15:07:10.475345 31021 raft_consensus.cc:2229] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:10.475379 31019 raft_consensus.cc:2258] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.475384 31021 raft_consensus.cc:2258] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.475695 31021 ts_tablet_manager.cc:1905] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.475705 31019 ts_tablet_manager.cc:1905] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.477260 31019 ts_tablet_manager.cc:1918] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:07:10.477321 31019 log.cc:1196] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/ac5921834108478eb9d2066292cbc834
I20260807 15:07:10.477537 31019 ts_tablet_manager.cc:1939] T ac5921834108478eb9d2066292cbc834 P 7ee6853fb1f44f21afaa25e67072871c: Deleting consensus metadata
I20260807 15:07:10.480799 31021 ts_tablet_manager.cc:1918] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:07:10.480877 31021 log.cc:1196] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/ac5921834108478eb9d2066292cbc834
I20260807 15:07:10.481086 31021 ts_tablet_manager.cc:1939] T ac5921834108478eb9d2066292cbc834 P f51cc4513040492d816536dc9db09fed: Deleting consensus metadata
I20260807 15:07:10.481433 30826 tablet_service.cc:1514] Processing DeleteTablet for tablet ac5921834108478eb9d2066292cbc834 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41676
I20260807 15:07:10.481544 31018 tablet_replica.cc:330] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: stopping tablet replica
I20260807 15:07:10.481606 31018 raft_consensus.cc:2229] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:10.481721 31018 raft_consensus.cc:2258] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.481837 30127 catalog_manager.cc:4841] TS f51cc4513040492d816536dc9db09fed (127.18.129.1:35383): tablet ac5921834108478eb9d2066292cbc834 (table test-table [id=a16b890c3638472c84073fad12d07268]) successfully deleted
I20260807 15:07:10.482152 30130 catalog_manager.cc:4841] TS 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023): tablet ac5921834108478eb9d2066292cbc834 (table test-table [id=a16b890c3638472c84073fad12d07268]) successfully deleted
I20260807 15:07:10.482425 31018 ts_tablet_manager.cc:1905] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.483866 31018 ts_tablet_manager.cc:1918] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:07:10.483930 31018 log.cc:1196] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/ac5921834108478eb9d2066292cbc834
I20260807 15:07:10.484135 31018 ts_tablet_manager.cc:1939] T ac5921834108478eb9d2066292cbc834 P b0e4c329706743efb1ba042c669a7d98: Deleting consensus metadata
I20260807 15:07:10.484450 30126 catalog_manager.cc:4841] TS b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485): tablet ac5921834108478eb9d2066292cbc834 (table test-table [id=a16b890c3638472c84073fad12d07268]) successfully deleted
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:07:10.509778 30234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:10.504474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.18.129.62:35845)
0807 15:07:10.504535 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.504537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.504656 (+ 119us) server_negotiation.cc:407] Connection header received
0807 15:07:10.504724 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.504726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.504737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.504783 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:10.505323 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.505830 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.506572 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.506764 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.506818 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.509709 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59392: BlockingRecv error: recv got EOF from 127.0.0.1:59392 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[30081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM
I20260807 15:07:10.517067 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.510552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59398 (local address 127.18.129.62:35845)
0807 15:07:10.510590 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.510592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.510780 (+ 188us) server_negotiation.cc:407] Connection header received
0807 15:07:10.510842 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.510843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.510880 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.510925 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:10.511361 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.511828 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.512457 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.512645 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.512697 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.514678 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:10.514680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:10.514681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:10.514688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:10.516100 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:10.516397 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:10.516398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:10.516398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:10.516452 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:10.516612 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:10.516613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:10.516614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:10.516720 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:10.516805 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.516900 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.516975 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:07:10.518471 30144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:07:10.518642 30144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:10.520051 30375 tablet_service.cc:1467] Processing CreateTablet for tablet 7cec93080ec94e86925bddcb4b01b04a (DEFAULT_TABLE table=test-table [id=b3b21b3e91854fab8e6220ba86b4d3c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.520049 30590 tablet_service.cc:1467] Processing CreateTablet for tablet 7cec93080ec94e86925bddcb4b01b04a (DEFAULT_TABLE table=test-table [id=b3b21b3e91854fab8e6220ba86b4d3c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.520190 30590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cec93080ec94e86925bddcb4b01b04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.520191 30375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cec93080ec94e86925bddcb4b01b04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.521484 30980 tablet_bootstrap.cc:492] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: Bootstrap starting.
I20260807 15:07:10.521566 30979 tablet_bootstrap.cc:492] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: Bootstrap starting.
I20260807 15:07:10.521888 30980 tablet_bootstrap.cc:654] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.521885 30979 tablet_bootstrap.cc:654] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.522154 30826 tablet_service.cc:1467] Processing CreateTablet for tablet 7cec93080ec94e86925bddcb4b01b04a (DEFAULT_TABLE table=test-table [id=b3b21b3e91854fab8e6220ba86b4d3c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.522266 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cec93080ec94e86925bddcb4b01b04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.522970 30980 tablet_bootstrap.cc:492] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: No bootstrap required, opened a new log
I20260807 15:07:10.523034 30980 ts_tablet_manager.cc:1397] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:10.523185 30980 raft_consensus.cc:348] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.523252 30980 raft_consensus.cc:374] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.523269 30980 raft_consensus.cc:729] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee6853fb1f44f21afaa25e67072871c, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.523309 30980 consensus_queue.cc:260] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.523389 30980 ts_tablet_manager.cc:1428] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.523742 30985 tablet_bootstrap.cc:492] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: Bootstrap starting.
I20260807 15:07:10.524159 30985 tablet_bootstrap.cc:654] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.525079 30985 tablet_bootstrap.cc:492] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: No bootstrap required, opened a new log
I20260807 15:07:10.525126 30985 ts_tablet_manager.cc:1397] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:07:10.525223 30979 tablet_bootstrap.cc:492] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: No bootstrap required, opened a new log
I20260807 15:07:10.525264 30979 ts_tablet_manager.cc:1397] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 15:07:10.525293 30985 raft_consensus.cc:348] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.525357 30985 raft_consensus.cc:374] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.525375 30985 raft_consensus.cc:729] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e4c329706743efb1ba042c669a7d98, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.525365 30979 raft_consensus.cc:348] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.525422 30979 raft_consensus.cc:374] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.525414 30985 consensus_queue.cc:260] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.525441 30979 raft_consensus.cc:729] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51cc4513040492d816536dc9db09fed, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.525493 30985 ts_tablet_manager.cc:1428] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.525476 30979 consensus_queue.cc:260] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.525632 30979 ts_tablet_manager.cc:1428] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.560767 31050 raft_consensus.cc:482] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:10.560861 31050 raft_consensus.cc:504] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.561028 31050 leader_election.cc:288] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f51cc4513040492d816536dc9db09fed (127.18.129.1:35383), 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023)
I20260807 15:07:10.561259 30625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cec93080ec94e86925bddcb4b01b04a" candidate_uuid: "b0e4c329706743efb1ba042c669a7d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ee6853fb1f44f21afaa25e67072871c" is_pre_election: true
I20260807 15:07:10.561247 30402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cec93080ec94e86925bddcb4b01b04a" candidate_uuid: "b0e4c329706743efb1ba042c669a7d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51cc4513040492d816536dc9db09fed" is_pre_election: true
I20260807 15:07:10.561343 30625 raft_consensus.cc:2454] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0e4c329706743efb1ba042c669a7d98 in term 0.
I20260807 15:07:10.561343 30402 raft_consensus.cc:2454] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0e4c329706743efb1ba042c669a7d98 in term 0.
I20260807 15:07:10.561504 30750 leader_election.cc:302] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7ee6853fb1f44f21afaa25e67072871c, b0e4c329706743efb1ba042c669a7d98, f51cc4513040492d816536dc9db09fed; no voters:
I20260807 15:07:10.561620 31050 raft_consensus.cc:2781] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:10.561650 31050 raft_consensus.cc:482] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:10.561666 31050 raft_consensus.cc:3037] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.562430 31050 raft_consensus.cc:504] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.562556 31050 leader_election.cc:288] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [CANDIDATE]: Term 1 election: Requested vote from peers f51cc4513040492d816536dc9db09fed (127.18.129.1:35383), 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023)
I20260807 15:07:10.562765 30402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cec93080ec94e86925bddcb4b01b04a" candidate_uuid: "b0e4c329706743efb1ba042c669a7d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51cc4513040492d816536dc9db09fed"
I20260807 15:07:10.562798 30625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cec93080ec94e86925bddcb4b01b04a" candidate_uuid: "b0e4c329706743efb1ba042c669a7d98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ee6853fb1f44f21afaa25e67072871c"
I20260807 15:07:10.562873 30402 raft_consensus.cc:3037] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.562884 30625 raft_consensus.cc:3037] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.563627 30625 raft_consensus.cc:2454] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0e4c329706743efb1ba042c669a7d98 in term 1.
I20260807 15:07:10.563627 30402 raft_consensus.cc:2454] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0e4c329706743efb1ba042c669a7d98 in term 1.
I20260807 15:07:10.563802 30748 leader_election.cc:302] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7ee6853fb1f44f21afaa25e67072871c, b0e4c329706743efb1ba042c669a7d98, f51cc4513040492d816536dc9db09fed; no voters:
I20260807 15:07:10.563912 31050 raft_consensus.cc:2781] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:10.563951 31050 raft_consensus.cc:686] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 1 LEADER]: Becoming Leader. State: Replica: b0e4c329706743efb1ba042c669a7d98, State: Running, Role: LEADER
I20260807 15:07:10.564002 31050 consensus_queue.cc:237] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.564546 30146 catalog_manager.cc:5495] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0e4c329706743efb1ba042c669a7d98 (127.18.129.3). New cstate: current_term: 1 leader_uuid: "b0e4c329706743efb1ba042c669a7d98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } health_report { overall_health: HEALTHY } } }
I20260807 15:07:10.569556 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.567426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41680 (local address 127.18.129.3:42485)
0807 15:07:10.567468 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.567470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.567511 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:07:10.567583 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.567584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.567594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.567630 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:10.567755 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.568222 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.568942 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.569144 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.569198 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.569327 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.569361 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.569403 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.569451 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.569473 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:07:10.570307 30625 raft_consensus.cc:1261] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Refusing update from remote peer b0e4c329706743efb1ba042c669a7d98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.570297 30402 raft_consensus.cc:1261] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Refusing update from remote peer b0e4c329706743efb1ba042c669a7d98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.570497 31050 consensus_queue.cc:1035] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.570693 31050 consensus_queue.cc:1035] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.574573 30144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:10.574669 30144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:10.575277 30144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: Sending DeleteTablet for 3 replicas of tablet 7cec93080ec94e86925bddcb4b01b04a
I20260807 15:07:10.575515 30826 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cec93080ec94e86925bddcb4b01b04a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41676
I20260807 15:07:10.575573 30375 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cec93080ec94e86925bddcb4b01b04a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:34810
I20260807 15:07:10.575659 31018 tablet_replica.cc:330] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: stopping tablet replica
I20260807 15:07:10.575724 31018 raft_consensus.cc:2229] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:07:10.575814 31018 raft_consensus.cc:2258] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.575891 31021 tablet_replica.cc:330] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: stopping tablet replica
I20260807 15:07:10.575953 31021 raft_consensus.cc:2229] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:10.575996 31021 raft_consensus.cc:2258] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.576260 31018 ts_tablet_manager.cc:1905] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.576469 31021 ts_tablet_manager.cc:1905] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.577847 31021 ts_tablet_manager.cc:1918] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:10.577848 30590 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cec93080ec94e86925bddcb4b01b04a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:52292
I20260807 15:07:10.577898 31021 log.cc:1196] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/7cec93080ec94e86925bddcb4b01b04a
I20260807 15:07:10.577925 31019 tablet_replica.cc:330] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: stopping tablet replica
I20260807 15:07:10.577961 31019 raft_consensus.cc:2229] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:10.577999 31019 raft_consensus.cc:2258] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.578116 31021 ts_tablet_manager.cc:1939] T 7cec93080ec94e86925bddcb4b01b04a P f51cc4513040492d816536dc9db09fed: Deleting consensus metadata
I20260807 15:07:10.578428 31018 ts_tablet_manager.cc:1918] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:10.578380 30144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:07:10.578478 31018 log.cc:1196] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/7cec93080ec94e86925bddcb4b01b04a
W20260807 15:07:10.578529 30144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:10.578680 31018 ts_tablet_manager.cc:1939] T 7cec93080ec94e86925bddcb4b01b04a P b0e4c329706743efb1ba042c669a7d98: Deleting consensus metadata
I20260807 15:07:10.578980 30126 catalog_manager.cc:4841] TS b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485): tablet 7cec93080ec94e86925bddcb4b01b04a (table test-table [id=b3b21b3e91854fab8e6220ba86b4d3c8]) successfully deleted
I20260807 15:07:10.579560 30127 catalog_manager.cc:4841] TS f51cc4513040492d816536dc9db09fed (127.18.129.1:35383): tablet 7cec93080ec94e86925bddcb4b01b04a (table test-table [id=b3b21b3e91854fab8e6220ba86b4d3c8]) successfully deleted
I20260807 15:07:10.579648 31019 ts_tablet_manager.cc:1905] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.580441 30375 tablet_service.cc:1467] Processing CreateTablet for tablet c06e85a66f39494c8cc9e6b851ee2951 (DEFAULT_TABLE table=test-table [id=4e3f218a885c4768bb6fa7c53a570d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.580540 30375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c06e85a66f39494c8cc9e6b851ee2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.581112 31019 ts_tablet_manager.cc:1918] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:10.581161 31019 log.cc:1196] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/7cec93080ec94e86925bddcb4b01b04a
I20260807 15:07:10.581341 31019 ts_tablet_manager.cc:1939] T 7cec93080ec94e86925bddcb4b01b04a P 7ee6853fb1f44f21afaa25e67072871c: Deleting consensus metadata
I20260807 15:07:10.581614 30590 tablet_service.cc:1467] Processing CreateTablet for tablet c06e85a66f39494c8cc9e6b851ee2951 (DEFAULT_TABLE table=test-table [id=4e3f218a885c4768bb6fa7c53a570d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.581704 30590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c06e85a66f39494c8cc9e6b851ee2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.582645 30980 tablet_bootstrap.cc:492] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: Bootstrap starting.
I20260807 15:07:10.582653 30979 tablet_bootstrap.cc:492] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed: Bootstrap starting.
I20260807 15:07:10.583032 30979 tablet_bootstrap.cc:654] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.583120 30980 tablet_bootstrap.cc:654] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.583297 30826 tablet_service.cc:1467] Processing CreateTablet for tablet c06e85a66f39494c8cc9e6b851ee2951 (DEFAULT_TABLE table=test-table [id=4e3f218a885c4768bb6fa7c53a570d72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:10.583395 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c06e85a66f39494c8cc9e6b851ee2951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:10.584000 30980 tablet_bootstrap.cc:492] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: No bootstrap required, opened a new log
I20260807 15:07:10.584055 30980 ts_tablet_manager.cc:1397] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:07:10.584215 30980 raft_consensus.cc:348] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.584278 30980 raft_consensus.cc:374] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.584296 30980 raft_consensus.cc:729] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee6853fb1f44f21afaa25e67072871c, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.584337 30980 consensus_queue.cc:260] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.584417 30980 ts_tablet_manager.cc:1428] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.584597 30985 tablet_bootstrap.cc:492] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: Bootstrap starting.
I20260807 15:07:10.584903 30985 tablet_bootstrap.cc:654] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:10.585285 30130 catalog_manager.cc:4841] TS 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023): tablet 7cec93080ec94e86925bddcb4b01b04a (table test-table [id=b3b21b3e91854fab8e6220ba86b4d3c8]) successfully deleted
I20260807 15:07:10.585799 30979 tablet_bootstrap.cc:492] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed: No bootstrap required, opened a new log
I20260807 15:07:10.585850 30979 ts_tablet_manager.cc:1397] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:07:10.585950 30979 raft_consensus.cc:348] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.586012 30979 raft_consensus.cc:374] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.586030 30979 raft_consensus.cc:729] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51cc4513040492d816536dc9db09fed, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.586069 30979 consensus_queue.cc:260] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.586151 30979 ts_tablet_manager.cc:1428] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.586683 30985 tablet_bootstrap.cc:492] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: No bootstrap required, opened a new log
I20260807 15:07:10.586732 30985 ts_tablet_manager.cc:1397] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:07:10.586897 30985 raft_consensus.cc:348] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.586966 30985 raft_consensus.cc:374] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:10.586983 30985 raft_consensus.cc:729] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e4c329706743efb1ba042c669a7d98, State: Initialized, Role: FOLLOWER
I20260807 15:07:10.587020 30985 consensus_queue.cc:260] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.587123 30985 ts_tablet_manager.cc:1428] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:10.659085 31030 raft_consensus.cc:482] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:10.659183 31030 raft_consensus.cc:504] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.659327 31030 leader_election.cc:288] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f51cc4513040492d816536dc9db09fed (127.18.129.1:35383), b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485)
I20260807 15:07:10.659538 30402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c06e85a66f39494c8cc9e6b851ee2951" candidate_uuid: "7ee6853fb1f44f21afaa25e67072871c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51cc4513040492d816536dc9db09fed" is_pre_election: true
I20260807 15:07:10.659526 30858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c06e85a66f39494c8cc9e6b851ee2951" candidate_uuid: "7ee6853fb1f44f21afaa25e67072871c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0e4c329706743efb1ba042c669a7d98" is_pre_election: true
I20260807 15:07:10.659641 30858 raft_consensus.cc:2454] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee6853fb1f44f21afaa25e67072871c in term 0.
I20260807 15:07:10.659641 30402 raft_consensus.cc:2454] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee6853fb1f44f21afaa25e67072871c in term 0.
I20260807 15:07:10.659787 30518 leader_election.cc:302] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ee6853fb1f44f21afaa25e67072871c, b0e4c329706743efb1ba042c669a7d98; no voters:
I20260807 15:07:10.659888 31030 raft_consensus.cc:2781] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:10.659924 31030 raft_consensus.cc:482] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:10.659957 31030 raft_consensus.cc:3037] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.660633 31030 raft_consensus.cc:504] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.660763 31030 leader_election.cc:288] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [CANDIDATE]: Term 1 election: Requested vote from peers f51cc4513040492d816536dc9db09fed (127.18.129.1:35383), b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485)
I20260807 15:07:10.660929 30858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c06e85a66f39494c8cc9e6b851ee2951" candidate_uuid: "7ee6853fb1f44f21afaa25e67072871c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0e4c329706743efb1ba042c669a7d98"
I20260807 15:07:10.660926 30402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c06e85a66f39494c8cc9e6b851ee2951" candidate_uuid: "7ee6853fb1f44f21afaa25e67072871c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f51cc4513040492d816536dc9db09fed"
I20260807 15:07:10.661007 30858 raft_consensus.cc:3037] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.661034 30402 raft_consensus.cc:3037] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:10.661716 30858 raft_consensus.cc:2454] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee6853fb1f44f21afaa25e67072871c in term 1.
I20260807 15:07:10.661870 30518 leader_election.cc:302] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ee6853fb1f44f21afaa25e67072871c, b0e4c329706743efb1ba042c669a7d98; no voters:
I20260807 15:07:10.661890 30402 raft_consensus.cc:2454] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee6853fb1f44f21afaa25e67072871c in term 1.
I20260807 15:07:10.661955 31030 raft_consensus.cc:2781] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:10.662001 31030 raft_consensus.cc:686] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 1 LEADER]: Becoming Leader. State: Replica: 7ee6853fb1f44f21afaa25e67072871c, State: Running, Role: LEADER
I20260807 15:07:10.662053 31030 consensus_queue.cc:237] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } }
I20260807 15:07:10.662644 30144 catalog_manager.cc:5495] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2). New cstate: current_term: 1 leader_uuid: "7ee6853fb1f44f21afaa25e67072871c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ee6853fb1f44f21afaa25e67072871c" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 } health_report { overall_health: UNKNOWN } } }
I20260807 15:07:10.679425 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.676593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.18.129.2:43023)
0807 15:07:10.676642 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:10.676644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:10.676651 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:07:10.676671 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:10.676672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:10.676682 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:10.676724 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:10.676845 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.677382 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.678454 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:10.678745 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:10.678819 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:10.679043 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:10.679080 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:10.679135 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:10.679315 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 15:07:10.679341 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:07:10.682487 30858 raft_consensus.cc:1261] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Refusing update from remote peer 7ee6853fb1f44f21afaa25e67072871c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.682487 30402 raft_consensus.cc:1261] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Refusing update from remote peer 7ee6853fb1f44f21afaa25e67072871c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:10.682713 30987 consensus_queue.cc:1035] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0e4c329706743efb1ba042c669a7d98" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 42485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.682873 30987 consensus_queue.cc:1035] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f51cc4513040492d816536dc9db09fed" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:10.693179 30146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:10.693275 30146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:10.693848 30146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 350ec1582ccb4162b9319d7e50547434: Sending DeleteTablet for 3 replicas of tablet c06e85a66f39494c8cc9e6b851ee2951
I20260807 15:07:10.694065 30375 tablet_service.cc:1514] Processing DeleteTablet for tablet c06e85a66f39494c8cc9e6b851ee2951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:34810
I20260807 15:07:10.694098 30590 tablet_service.cc:1514] Processing DeleteTablet for tablet c06e85a66f39494c8cc9e6b851ee2951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:52292
I20260807 15:07:10.694183 31019 tablet_replica.cc:330] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: stopping tablet replica
I20260807 15:07:10.694188 31021 tablet_replica.cc:330] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed: stopping tablet replica
I20260807 15:07:10.694233 31019 raft_consensus.cc:2229] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:07:10.694243 31021 raft_consensus.cc:2229] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:10.694275 31021 raft_consensus.cc:2258] T c06e85a66f39494c8cc9e6b851ee2951 P f51cc4513040492d816536dc9db09fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.694285 31019 raft_consensus.cc:2258] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.694532 31019 ts_tablet_manager.cc:1905] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.694562 30826 tablet_service.cc:1514] Processing DeleteTablet for tablet c06e85a66f39494c8cc9e6b851ee2951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:10 UTC) from {username='oryx', principal='oryx/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41676
I20260807 15:07:10.694638 31018 tablet_replica.cc:330] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: stopping tablet replica
I20260807 15:07:10.694684 31018 raft_consensus.cc:2229] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:10.694723 31018 raft_consensus.cc:2258] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:10.694927 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 30268
I20260807 15:07:10.696200 31019 ts_tablet_manager.cc:1918] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:07:10.696271 31019 log.cc:1196] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/c06e85a66f39494c8cc9e6b851ee2951
I20260807 15:07:10.696491 31019 ts_tablet_manager.cc:1939] T c06e85a66f39494c8cc9e6b851ee2951 P 7ee6853fb1f44f21afaa25e67072871c: Deleting consensus metadata
I20260807 15:07:10.705271 30130 catalog_manager.cc:4841] TS 7ee6853fb1f44f21afaa25e67072871c (127.18.129.2:43023): tablet c06e85a66f39494c8cc9e6b851ee2951 (table test-table [id=4e3f218a885c4768bb6fa7c53a570d72]) successfully deleted
W20260807 15:07:10.711475 30127 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:35383 (error 108)
I20260807 15:07:10.711679 31018 ts_tablet_manager.cc:1905] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:10.712193 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 30487
I20260807 15:07:10.718333 31018 ts_tablet_manager.cc:1918] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:07:10.718418 31018 log.cc:1196] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/c06e85a66f39494c8cc9e6b851ee2951
I20260807 15:07:10.718636 31018 ts_tablet_manager.cc:1939] T c06e85a66f39494c8cc9e6b851ee2951 P b0e4c329706743efb1ba042c669a7d98: Deleting consensus metadata
I20260807 15:07:10.718899 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:10.712389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35383 (local address 127.0.0.1:34830)
0807 15:07:10.718755 (+ 6366us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:10.718824 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:35383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6345,"thread_start_us":43,"threads_started":1}
W20260807 15:07:10.719007 30127 catalog_manager.cc:4584] TS f51cc4513040492d816536dc9db09fed (127.18.129.1:35383): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c06e85a66f39494c8cc9e6b851ee2951: Network error: Client connection negotiation failed: client connection to 127.18.129.1:35383: connect: Connection refused (error 111)
I20260807 15:07:10.719102 30126 catalog_manager.cc:4841] TS b0e4c329706743efb1ba042c669a7d98 (127.18.129.3:42485): tablet c06e85a66f39494c8cc9e6b851ee2951 (table test-table [id=4e3f218a885c4768bb6fa7c53a570d72]) successfully deleted
I20260807 15:07:10.725494 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 30716
I20260807 15:07:10.739529 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 30101
2026-08-07T15:07:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2632 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[31124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[31124](info): set up 2 sockets
Aug 07 15:07:10 dist-test-slave-5bn2 krb5kdc[31124](info): commencing operation
krb5kdc: starting...
W20260807 15:07:11.876266 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:07:11 dist-test-slave-5bn2 krb5kdc[31124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:07:11.997087 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:41349
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:41349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:12.078903 31142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:12.079125 31142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:12.079159 31142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:12.080324 31142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:12.080387 31142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:12.080410 31142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:12.080430 31142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:12.080447 31142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:12.081952 31142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:41349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:41349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:12.082223 31142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:12.082458 31142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:12.087347 31156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:12.087378 31151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:12.087464 31142 server_base.cc:1034] running on GCE node
W20260807 15:07:12.087361 31152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:12.087842 31142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:12.088109 31142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:12.089259 31142 hybrid_clock.cc:648] HybridClock initialized: now 1786115232089246 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T15:07:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1347 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:07:12 dist-test-slave-5bn2 krb5kdc[31164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:07:12 dist-test-slave-5bn2 krb5kdc[31164](info): set up 2 sockets
Aug 07 15:07:12 dist-test-slave-5bn2 krb5kdc[31164](info): commencing operation
krb5kdc: starting...
W20260807 15:07:13.224620 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:07:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:07:13.342555 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39963
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:39963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:13.423923 31185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:13.424121 31185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:13.424142 31185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:13.425264 31185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:13.425305 31185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:13.425318 31185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:13.425341 31185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:13.425364 31185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:13.426870 31185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:39963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:13.427101 31185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:13.427312 31185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:13.432119 31192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:13.432145 31194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:13.432188 31185 server_base.cc:1034] running on GCE node
W20260807 15:07:13.432119 31198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:13.432521 31185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:13.432824 31185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:13.433979 31185 hybrid_clock.cc:648] HybridClock initialized: now 1786115233433963 us; error 28 us; skew 500 ppm
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:13.441937 31185 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:07:13.442420 31185 webserver.cc:460] Webserver started at http://127.18.129.62:38853/ using document root <none> and password file <none>
I20260807 15:07:13.442718 31185 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:13.442782 31185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:13.442951 31185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:13.444046 31185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "78d47fc8b343424296f7e786de63b050"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "eecd2f5e77481016545a6322c65f52fb"
server_key_iv: "1387cc6c1266dd6c3b12b57c4cfa6bda"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.444387 31185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "78d47fc8b343424296f7e786de63b050"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "eecd2f5e77481016545a6322c65f52fb"
server_key_iv: "1387cc6c1266dd6c3b12b57c4cfa6bda"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.445652 31185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:13.446333 31210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.446493 31185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:13.446590 31185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "78d47fc8b343424296f7e786de63b050"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "eecd2f5e77481016545a6322c65f52fb"
server_key_iv: "1387cc6c1266dd6c3b12b57c4cfa6bda"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.446662 31185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:13.464035 31185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:13.465760 31185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:13.465893 31185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:13.469779 31185 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:39963
I20260807 15:07:13.469817 31294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:39963 every 8 connection(s)
I20260807 15:07:13.470165 31185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:13.471038 31295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:13.473335 31295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Bootstrap starting.
I20260807 15:07:13.473907 31295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:13.474171 31295 log.cc:824] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:13.474824 31295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: No bootstrap required, opened a new log
I20260807 15:07:13.476451 31295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } }
I20260807 15:07:13.476604 31295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:13.476639 31295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d47fc8b343424296f7e786de63b050, State: Initialized, Role: FOLLOWER
I20260807 15:07:13.476783 31295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } }
I20260807 15:07:13.476858 31295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:13.476894 31295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:13.476941 31295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:13.477113 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 31185
I20260807 15:07:13.477223 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:13.477375 18948 external_mini_cluster.cc:1428] Setting key c4e705745d623a3c7e704908ec7578d1
I20260807 15:07:13.477550 31295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } }
I20260807 15:07:13.477660 31295 leader_election.cc:302] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78d47fc8b343424296f7e786de63b050; no voters:
I20260807 15:07:13.477840 31295 leader_election.cc:288] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:13.477888 31300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:13.478015 31300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 1 LEADER]: Becoming Leader. State: Replica: 78d47fc8b343424296f7e786de63b050, State: Running, Role: LEADER
I20260807 15:07:13.478098 31295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:13.478107 31300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } }
I20260807 15:07:13.478441 31300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d47fc8b343424296f7e786de63b050. Latest consensus state: current_term: 1 leader_uuid: "78d47fc8b343424296f7e786de63b050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } } }
I20260807 15:07:13.478516 31300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:13.478631 31303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78d47fc8b343424296f7e786de63b050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } } }
I20260807 15:07:13.478713 31303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:13.478786 31309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:13.479288 31309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:13.482018 31309 catalog_manager.cc:1347] Generated new cluster ID: 9309f346d0da4754b5a3b7f41882aaac
I20260807 15:07:13.482072 31309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:13.489702 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:13.477820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53116 (local address 127.18.129.62:39963)
0807 15:07:13.478208 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:13.478241 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:13.478279 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:07:13.478402 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:13.478410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:13.478470 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:13.479911 (+ 1441us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:13.480012 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.481425 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.482436 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.483087 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.483200 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:13.485816 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:13.485821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:13.485821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:13.485834 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:13.487950 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:13.488324 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:13.488327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:13.488327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:13.488468 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:13.488639 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:13.488640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:13.488640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:13.488960 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:13.489127 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:13.489197 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:07:13.489481 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":87,"threads_started":1}
I20260807 15:07:13.490548 31309 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:13.491261 31309 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:13.500957 31309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Generated new TSK 0
I20260807 15:07:13.501137 31309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:07:13.527280 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39963
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:13.608381 31335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:13.608599 31335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:13.608632 31335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:13.609779 31335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:13.609819 31335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:13.609853 31335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:13.611332 31335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:13.611558 31335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:13.611771 31335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:13.616993 31345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:13.617092 31349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:13.617000 31346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:13.617137 31335 server_base.cc:1034] running on GCE node
I20260807 15:07:13.617347 31335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:13.617537 31335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:13.618661 31335 hybrid_clock.cc:648] HybridClock initialized: now 1786115233618650 us; error 24 us; skew 500 ppm
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:13.621100 31335 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:07:13.621497 31335 webserver.cc:460] Webserver started at http://127.18.129.1:46697/ using document root <none> and password file <none>
I20260807 15:07:13.621697 31335 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:13.621749 31335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:13.621874 31335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:13.622949 31335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "da49e9aabb51486a8848dda9c47f3a31"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "65d468e30a64829e2e99ec53ab640bcf"
server_key_iv: "f63ff1655ef8401715b3168aa1b95176"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.623297 31335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "da49e9aabb51486a8848dda9c47f3a31"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "65d468e30a64829e2e99ec53ab640bcf"
server_key_iv: "f63ff1655ef8401715b3168aa1b95176"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.624538 31335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:13.625198 31360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.625344 31335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.625429 31335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "da49e9aabb51486a8848dda9c47f3a31"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "65d468e30a64829e2e99ec53ab640bcf"
server_key_iv: "f63ff1655ef8401715b3168aa1b95176"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.625490 31335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:13.641294 31335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:13.643064 31335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:13.643200 31335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:13.643404 31335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:13.643697 31335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:13.643729 31335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.643759 31335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:13.643782 31335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.649371 31335 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:41655
I20260807 15:07:13.649379 31540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:41655 every 8 connection(s)
I20260807 15:07:13.649715 31335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:07:13.652559 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 31335
I20260807 15:07:13.652650 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:07:13.652818 18948 external_mini_cluster.cc:1428] Setting key 4ffe42c9204ea8b404b3c679814e21e5
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:13.658633 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:13.650376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:48763 (local address 127.18.129.62:39963)
0807 15:07:13.650419 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:13.650421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:13.650610 (+ 189us) server_negotiation.cc:407] Connection header received
0807 15:07:13.651166 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:13.651167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:13.651184 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:13.651235 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:13.652168 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.652706 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.653829 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.654038 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.654099 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:13.656172 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:13.656173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:13.656174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:13.656181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:13.657361 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:13.657732 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:13.657733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:13.657734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:13.657793 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:13.658011 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:13.658012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:13.658013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:13.658126 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:13.658185 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:13.658442 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 15:07:13.658542 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:07:13.658829 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:13.650351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:48763)
0807 15:07:13.650491 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:13.650528 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:13.651064 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:13.651306 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:13.651310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:13.651869 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:13.652070 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:13.652075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.652789 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.652790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:13.653743 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:13.653746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.654114 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.654115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:13.654247 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:13.654713 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:13.654721 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:13.656086 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:13.657434 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:13.657441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:13.657443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:13.657664 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:13.657849 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:13.657850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:13.657850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:13.657957 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:13.658181 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:13.658188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:13.658325 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 15:07:13.658411 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:07:13.658568 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260807 15:07:13.659343 31541 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39963
I20260807 15:07:13.659457 31541 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:13.659660 31541 heartbeater.cc:511] Master 127.18.129.62:39963 requested a full tablet report, sending...
I20260807 15:07:13.660156 31236 ts_manager.cc:194] Registered new tserver with Master: da49e9aabb51486a8848dda9c47f3a31 (127.18.129.1:41655)
I20260807 15:07:13.660754 31236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:48763
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:07:13.677310 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39963
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:13.757611 31548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:13.757810 31548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:13.757833 31548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:13.758996 31548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:13.759034 31548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:13.759061 31548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:13.760483 31548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:13.760702 31548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:13.760915 31548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:13.765945 31559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:13.765933 31557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:13.765934 31563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:13.765971 31548 server_base.cc:1034] running on GCE node
I20260807 15:07:13.766405 31548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:13.766620 31548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:13.767755 31548 hybrid_clock.cc:648] HybridClock initialized: now 1786115233767741 us; error 27 us; skew 500 ppm
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:13.770184 31548 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:07:13.770560 31548 webserver.cc:460] Webserver started at http://127.18.129.2:43089/ using document root <none> and password file <none>
I20260807 15:07:13.770756 31548 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:13.770795 31548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:13.770931 31548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:13.771956 31548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "95780787cff94331a4ba0e0f051f6dc0"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "62b9f352d755cb93f9a56ba08ab3f85e"
server_key_iv: "f712d49b8d720b192d301ab1300ae285"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.772256 31548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "95780787cff94331a4ba0e0f051f6dc0"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "62b9f352d755cb93f9a56ba08ab3f85e"
server_key_iv: "f712d49b8d720b192d301ab1300ae285"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.773434 31548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:07:13.774060 31573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.774240 31548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:13.774324 31548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "95780787cff94331a4ba0e0f051f6dc0"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "62b9f352d755cb93f9a56ba08ab3f85e"
server_key_iv: "f712d49b8d720b192d301ab1300ae285"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.774376 31548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:13.803337 31548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:13.805440 31548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:13.805555 31548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:13.805752 31548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:13.806013 31548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:13.806044 31548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.806066 31548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:13.806085 31548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.811442 31548 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:41799
I20260807 15:07:13.811515 31741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:41799 every 8 connection(s)
I20260807 15:07:13.811800 31548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:07:13.812286 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 31548
I20260807 15:07:13.812388 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:07:13.812558 18948 external_mini_cluster.cc:1428] Setting key 4893d978fd7fe1b9d38f418aa099d274
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
I20260807 15:07:13.821641 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:13.812631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:36281 (local address 127.18.129.62:39963)
0807 15:07:13.812686 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:13.812688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:13.812880 (+ 192us) server_negotiation.cc:407] Connection header received
0807 15:07:13.813504 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:13.813505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:13.813518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:13.813584 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:13.814533 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.815258 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.816639 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.816832 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.816890 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:13.819176 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:13.819178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:13.819178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:13.819185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:13.820382 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:13.820749 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:13.820750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:13.820751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:13.820809 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:13.821025 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:13.821027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:13.821027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:13.821137 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:13.821194 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:13.821451 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 15:07:13.821548 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:07:13.821792 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:13.812465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:36281)
0807 15:07:13.812750 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:13.812798 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:13.813409 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:13.813640 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:13.813643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:13.814210 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:13.814437 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:13.814443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.815373 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.815374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:13.816553 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:13.816557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.816917 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.816918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:13.817052 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:13.817534 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:13.817544 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:13.819092 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:13.820456 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:13.820463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:13.820465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:13.820681 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:13.820865 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:13.820866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:13.820866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:13.820971 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:13.821194 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:13.821201 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:13.821335 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 15:07:13.821422 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:07:13.821570 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":202,"threads_started":1}
I20260807 15:07:13.822187 31742 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39963
I20260807 15:07:13.822273 31742 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:13.822438 31742 heartbeater.cc:511] Master 127.18.129.62:39963 requested a full tablet report, sending...
I20260807 15:07:13.822899 31236 ts_manager.cc:194] Registered new tserver with Master: 95780787cff94331a4ba0e0f051f6dc0 (127.18.129.2:41799)
I20260807 15:07:13.823271 31236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:36281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:07:13.838744 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39963
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:13.922361 31749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:13.922585 31749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:13.922618 31749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:13.923815 31749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:13.923858 31749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:13.923893 31749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:13.925302 31749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:13.925532 31749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:13.925745 31749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:13.930756 31757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:13.930824 31756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:13.930873 31759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:13.930962 31749 server_base.cc:1034] running on GCE node
I20260807 15:07:13.931169 31749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:13.931387 31749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:13.932525 31749 hybrid_clock.cc:648] HybridClock initialized: now 1786115233932508 us; error 31 us; skew 500 ppm
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:13.935002 31749 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:07:13.935413 31749 webserver.cc:460] Webserver started at http://127.18.129.3:43237/ using document root <none> and password file <none>
I20260807 15:07:13.935627 31749 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:13.935675 31749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:13.935806 31749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:13.936851 31749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "53deb11729c44fb19cbf7928a37f9723"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "05c88048ee26ed04227af16a8930cc51"
server_key_iv: "2c1351ff6899ee884c727134cea9f759"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.937196 31749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "53deb11729c44fb19cbf7928a37f9723"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "05c88048ee26ed04227af16a8930cc51"
server_key_iv: "2c1351ff6899ee884c727134cea9f759"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.938409 31749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:13.939097 31768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.939229 31749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:13.939316 31749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "53deb11729c44fb19cbf7928a37f9723"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "05c88048ee26ed04227af16a8930cc51"
server_key_iv: "2c1351ff6899ee884c727134cea9f759"
server_key_version: "encryptionkey@0"
I20260807 15:07:13.939383 31749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:13.951882 31749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:13.953919 31749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:13.954052 31749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:13.954262 31749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:13.954559 31749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:13.954594 31749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.954622 31749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:13.954645 31749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:13.960201 31749 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:38997
I20260807 15:07:13.960254 31952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:38997 every 8 connection(s)
I20260807 15:07:13.960546 31749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:07:13.963609 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 31749
I20260807 15:07:13.963732 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:07:13.963922 18948 external_mini_cluster.cc:1428] Setting key 2fe2aa62c40cc72e0850db40a31ae67b
Aug 07 15:07:13 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115233, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:13.969036 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:13.961189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:47275 (local address 127.18.129.62:39963)
0807 15:07:13.961251 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:13.961253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:13.961435 (+ 182us) server_negotiation.cc:407] Connection header received
0807 15:07:13.961914 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:13.961915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:13.961924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:13.961976 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:13.963006 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.963514 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.964703 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.964959 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.965014 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:13.967018 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:13.967019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:13.967019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:13.967025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:13.968041 (+ 1016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:13.968371 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:13.968372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:13.968372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:13.968418 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:13.968593 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:13.968594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:13.968594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:13.968684 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:13.968728 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:13.968919 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 15:07:13.968974 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:07:13.969238 31955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:13.961161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:47275)
0807 15:07:13.961311 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:13.961353 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:13.961834 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:13.962030 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:13.962041 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:13.962673 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:13.962932 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:13.962940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.963591 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.963592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:13.964638 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:13.964641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:13.965027 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:13.965027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:13.965139 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:13.965579 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:13.965587 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:13.966942 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:13.968103 (+ 1161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:13.968109 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:13.968111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:13.968317 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:13.968460 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:13.968460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:13.968461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:13.968549 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:13.968723 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:13.968726 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:13.968833 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 15:07:13.968906 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:07:13.969017 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260807 15:07:13.969661 31953 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39963
I20260807 15:07:13.969748 31953 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:13.969903 31953 heartbeater.cc:511] Master 127.18.129.62:39963 requested a full tablet report, sending...
I20260807 15:07:13.970258 31236 ts_manager.cc:194] Registered new tserver with Master: 53deb11729c44fb19cbf7928a37f9723 (127.18.129.3:38997)
I20260807 15:07:13.970613 31236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:47275
I20260807 15:07:13.975441 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab.
Entry for principal oryx/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab.
WARNING: no policy specified for oryx/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab.
Entry for principal oryx/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab.
WARNING: no policy specified for oryx/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab.
Entry for principal oryx/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab.
I20260807 15:07:14.043433 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 31335
I20260807 15:07:14.050206 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 31548
I20260807 15:07:14.063730 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 31749
I20260807 15:07:14.076840 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 31185
I20260807 15:07:14.085727 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39963
--webserver_interface=127.18.129.62
--webserver_port=38853
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:39963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:14.166356 31969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:14.166553 31969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:14.166581 31969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:14.167780 31969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:14.167825 31969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:14.167838 31969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:14.167851 31969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:14.167876 31969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:14.169358 31969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:39963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=38853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:14.169603 31969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:14.169816 31969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:14.174490 31982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:14.174520 31980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:14.174554 31969 server_base.cc:1034] running on GCE node
W20260807 15:07:14.174501 31979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:14.175009 31969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:14.175256 31969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:14.176379 31969 hybrid_clock.cc:648] HybridClock initialized: now 1786115234176367 us; error 30 us; skew 500 ppm
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115234, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:14.178732 31969 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:07:14.179167 31969 webserver.cc:460] Webserver started at http://127.18.129.62:38853/ using document root <none> and password file <none>
I20260807 15:07:14.179371 31969 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:14.179425 31969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:14.180666 31969 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:07:14.181249 31996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.181402 31969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:07:14.181468 31969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "78d47fc8b343424296f7e786de63b050"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "eecd2f5e77481016545a6322c65f52fb"
server_key_iv: "1387cc6c1266dd6c3b12b57c4cfa6bda"
server_key_version: "encryptionkey@0"
I20260807 15:07:14.181804 31969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:14.205875 31969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:14.207621 31969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:14.207748 31969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:14.211532 31969 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:39963
I20260807 15:07:14.211566 32085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:39963 every 8 connection(s)
I20260807 15:07:14.211910 31969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:14.213318 32088 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:07:14.213971 32088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Bootstrap starting.
I20260807 15:07:14.220569 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 31969
I20260807 15:07:14.220815 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:41655
--local_ip_for_outbound_sockets=127.18.129.1
--tserver_master_addrs=127.18.129.62:39963
--webserver_port=46697
--webserver_interface=127.18.129.1
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.129.1
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:07:14.222975 32088 log.cc:824] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:14.224591 32088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:07:14.224848 32088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Bootstrap complete.
I20260807 15:07:14.227051 32088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } }
I20260807 15:07:14.227283 32088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d47fc8b343424296f7e786de63b050, State: Initialized, Role: FOLLOWER
I20260807 15:07:14.227452 32088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } }
I20260807 15:07:14.227530 32088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:14.227591 32088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:14.227639 32088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:07:14.228621 32088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } }
I20260807 15:07:14.228719 32088 leader_election.cc:302] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78d47fc8b343424296f7e786de63b050; no voters:
I20260807 15:07:14.228895 32088 leader_election.cc:288] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:07:14.228946 32092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:07:14.229101 32092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [term 2 LEADER]: Becoming Leader. State: Replica: 78d47fc8b343424296f7e786de63b050, State: Running, Role: LEADER
I20260807 15:07:14.229171 32088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:14.229282 32092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } }
I20260807 15:07:14.230064 32094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d47fc8b343424296f7e786de63b050. Latest consensus state: current_term: 2 leader_uuid: "78d47fc8b343424296f7e786de63b050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } } }
I20260807 15:07:14.230149 32094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:14.230330 32107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:14.230790 32093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "78d47fc8b343424296f7e786de63b050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78d47fc8b343424296f7e786de63b050" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39963 } } }
I20260807 15:07:14.230842 32107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:14.231169 32107 catalog_manager.cc:1259] Loaded cluster ID: 9309f346d0da4754b5a3b7f41882aaac
I20260807 15:07:14.231218 32107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:14.232435 32107 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:14.232597 32107 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050: Loaded TSK: 0
I20260807 15:07:14.232792 32107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:07:14.232914 32093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d47fc8b343424296f7e786de63b050 [sys.catalog]: This master's current role is: LEADER
W20260807 15:07:14.305074 32090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:14.305286 32090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:14.305318 32090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:14.306502 32090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:14.306546 32090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:14.306581 32090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:14.308113 32090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.1.keytab
--principal=oryx/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:41655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=46697
--tserver_master_addrs=127.18.129.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:14.308350 32090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:14.308574 32090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:14.313716 32133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:14.313828 32135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:14.313742 32126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:14.314204 32090 server_base.cc:1034] running on GCE node
I20260807 15:07:14.314405 32090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:14.314632 32090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:14.315794 32090 hybrid_clock.cc:648] HybridClock initialized: now 1786115234315776 us; error 30 us; skew 500 ppm
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115234, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:14.318930 32090 init.cc:377] Logged in from keytab as oryx/127.18.129.1@KRBTEST.COM (short username oryx)
I20260807 15:07:14.319501 32090 webserver.cc:460] Webserver started at http://127.18.129.1:46697/ using document root <none> and password file <none>
I20260807 15:07:14.319765 32090 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:14.319872 32090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:14.321707 32090 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:07:14.322405 32142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.322579 32090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:14.322685 32090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "da49e9aabb51486a8848dda9c47f3a31"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "65d468e30a64829e2e99ec53ab640bcf"
server_key_iv: "f63ff1655ef8401715b3168aa1b95176"
server_key_version: "encryptionkey@0"
I20260807 15:07:14.323184 32090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:14.345252 32090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:14.347721 32090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:14.347921 32090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:14.348204 32090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:14.348567 32090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:14.348618 32090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.348685 32090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:14.348711 32090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.356628 32090 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:41655
I20260807 15:07:14.356669 32315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:41655 every 8 connection(s)
I20260807 15:07:14.357151 32090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:14.365492 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.357886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:44169)
0807 15:07:14.358056 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.358120 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.358616 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.359066 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.359079 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.359646 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.359840 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.359846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.360932 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.360933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.361743 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.361747 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.362323 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.362325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.362445 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.363051 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.363059 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.365282 (+ 2223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
W20260807 15:07:14.365581 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.357967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:44169 (local address 127.18.129.62:39963)
0807 15:07:14.358165 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.358173 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.358219 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:07:14.358807 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.358812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.358889 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.359012 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.359965 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.360828 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.361832 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.362237 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.362332 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.365424 (+ 3092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:44169: BlockingRecv error: recv got EOF from 127.18.129.1:44169 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
W20260807 15:07:14.365777 32317 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:39963 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.129.62:39963: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
I20260807 15:07:14.366564 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 32090
I20260807 15:07:14.366840 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:41799
--local_ip_for_outbound_sockets=127.18.129.2
--tserver_master_addrs=127.18.129.62:39963
--webserver_port=43089
--webserver_interface=127.18.129.2
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.129.2
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:14.369907 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.365978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:42987)
0807 15:07:14.366020 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.366030 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.366067 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.366185 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.366185 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.366308 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.366349 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.366350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.367034 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.367035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.367535 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.367536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.367868 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.367869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.367931 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.368342 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.368348 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.369862 (+ 1514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 15:07:14.370081 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.366046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:42987 (local address 127.18.129.62:39963)
0807 15:07:14.366093 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.366095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.366103 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:14.366124 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.366125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.366136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.366178 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.366417 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.366948 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.367613 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.367810 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.367864 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.370019 (+ 2155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:42987: BlockingRecv error: recv got EOF from 127.18.129.1:42987 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:14.374418 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.370152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:32927)
0807 15:07:14.370276 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.370284 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.370321 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.370476 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.370477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.370577 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.370604 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.370605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.371448 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.371450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.371950 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.371952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.372421 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.372422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.372470 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.372863 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.372867 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.374355 (+ 1488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110}
W20260807 15:07:14.374477 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.370241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:32927 (local address 127.18.129.62:39963)
0807 15:07:14.370275 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.370276 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.370313 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:07:14.370359 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.370359 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.370369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.370422 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.370675 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.371361 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.372032 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.372306 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.372375 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.374427 (+ 2052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:32927: BlockingRecv error: recv got EOF from 127.18.129.1:32927 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 15:07:14.374537 32317 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:07:14.452662 32324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:14.452867 32324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:14.452889 32324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:14.453972 32324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:14.454003 32324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:14.454028 32324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:14.455439 32324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.2.keytab
--principal=oryx/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:41799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=43089
--tserver_master_addrs=127.18.129.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:14.455623 32324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:14.455793 32324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:14.460708 32335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:14.460717 32339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:14.460999 32336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:14.461030 32324 server_base.cc:1034] running on GCE node
I20260807 15:07:14.461263 32324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:14.461479 32324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:14.462621 32324 hybrid_clock.cc:648] HybridClock initialized: now 1786115234462605 us; error 28 us; skew 500 ppm
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115234, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:14.465132 32324 init.cc:377] Logged in from keytab as oryx/127.18.129.2@KRBTEST.COM (short username oryx)
I20260807 15:07:14.465580 32324 webserver.cc:460] Webserver started at http://127.18.129.2:43089/ using document root <none> and password file <none>
I20260807 15:07:14.465770 32324 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:14.465824 32324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:14.467092 32324 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:07:14.467612 32349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.467767 32324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.467849 32324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "95780787cff94331a4ba0e0f051f6dc0"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "62b9f352d755cb93f9a56ba08ab3f85e"
server_key_iv: "f712d49b8d720b192d301ab1300ae285"
server_key_version: "encryptionkey@0"
I20260807 15:07:14.468186 32324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:14.483705 32324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:14.485617 32324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:14.485739 32324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:14.485948 32324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:14.486256 32324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:14.486290 32324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.486320 32324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:14.486343 32324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.492127 32324 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:41799
I20260807 15:07:14.492220 32540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:41799 every 8 connection(s)
I20260807 15:07:14.492537 32324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:07:14.494417 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 32324
I20260807 15:07:14.494727 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:38997
--local_ip_for_outbound_sockets=127.18.129.3
--tserver_master_addrs=127.18.129.62:39963
--webserver_port=43237
--webserver_interface=127.18.129.3
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.129.3
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:07:14.500458 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.493226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:36829 (local address 127.18.129.62:39963)
0807 15:07:14.493285 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.493287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.493481 (+ 194us) server_negotiation.cc:407] Connection header received
0807 15:07:14.494000 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.494001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.494018 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.494077 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.495316 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.495806 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.497089 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.497363 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.497436 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.500407 (+ 2971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:36829: BlockingRecv error: recv got EOF from 127.18.129.2:36829 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:07:14.500545 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.493212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:36829)
0807 15:07:14.493368 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.493404 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.493902 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.494148 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.494151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.494951 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.495216 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.495223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.495897 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.495898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.496948 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.496951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.497439 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.497439 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.497555 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.498026 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.498035 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.500330 (+ 2295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
W20260807 15:07:14.500785 32541 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:39963 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.129.62:39963: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:14.505244 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.500998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:38805)
0807 15:07:14.501046 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.501058 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.501095 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.501234 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.501235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.501345 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.501389 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.501390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.502211 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.502212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.502690 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.502691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.503138 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.503139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.503186 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.503583 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.503587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.505195 (+ 1608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 15:07:14.505365 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.501062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:38805 (local address 127.18.129.62:39963)
0807 15:07:14.501121 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.501122 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.501130 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:14.501155 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.501156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.501167 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.501226 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.501446 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.502139 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.502759 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.503056 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.503144 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.505302 (+ 2158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:38805: BlockingRecv error: recv got EOF from 127.18.129.2:38805 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:14.509758 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.505459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:49815)
0807 15:07:14.505635 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.505645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.505700 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.505846 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.505846 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.505985 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.506044 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.506046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.506628 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.506629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.507223 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.507225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.507549 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.507550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.507602 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.508005 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.508009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.509678 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
W20260807 15:07:14.509886 32541 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:07:14.509821 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.505705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:49815 (local address 127.18.129.62:39963)
0807 15:07:14.505759 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.505760 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.505767 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:07:14.505785 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.505786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.505793 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.505835 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.506110 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.506564 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.507294 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.507471 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.507523 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.509767 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:49815: BlockingRecv error: recv got EOF from 127.18.129.2:49815 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 15:07:14.577973 32545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:14.578184 32545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:14.578207 32545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:14.579375 32545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:14.579416 32545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:14.579444 32545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:14.580902 32545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/krb5kdc/oryx_127.18.129.3.keytab
--principal=oryx/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:38997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=43237
--tserver_master_addrs=127.18.129.62:39963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:14.581130 32545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:14.581346 32545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:14.586282 32558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:14.586299 32563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:14.586282 32554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:14.586375 32545 server_base.cc:1034] running on GCE node
I20260807 15:07:14.586663 32545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:14.586910 32545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:14.588048 32545 hybrid_clock.cc:648] HybridClock initialized: now 1786115234588035 us; error 27 us; skew 500 ppm
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115234, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:14.590428 32545 init.cc:377] Logged in from keytab as oryx/127.18.129.3@KRBTEST.COM (short username oryx)
I20260807 15:07:14.590838 32545 webserver.cc:460] Webserver started at http://127.18.129.3:43237/ using document root <none> and password file <none>
I20260807 15:07:14.591120 32545 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:14.591262 32545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:14.592605 32545 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:07:14.593171 32571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.593322 32545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:07:14.593407 32545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "53deb11729c44fb19cbf7928a37f9723"
format_stamp: "Formatted at 2026-08-07 15:07:13 on dist-test-slave-5bn2"
server_key: "05c88048ee26ed04227af16a8930cc51"
server_key_iv: "2c1351ff6899ee884c727134cea9f759"
server_key_version: "encryptionkey@0"
I20260807 15:07:14.593736 32545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:14.601974 32545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:14.603668 32545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:14.603802 32545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:14.604007 32545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:14.604336 32545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:14.604372 32545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.604399 32545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:14.604417 32545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:14.610075 32545 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:38997
I20260807 15:07:14.610177 32759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:38997 every 8 connection(s)
I20260807 15:07:14.610473 32545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:07:14.617175 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.611163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:53179 (local address 127.18.129.62:39963)
0807 15:07:14.611217 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.611219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.611412 (+ 193us) server_negotiation.cc:407] Connection header received
0807 15:07:14.611869 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.611870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.611879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.611925 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.612794 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.613273 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.614161 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.614340 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.614387 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.617126 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:53179: BlockingRecv error: recv got EOF from 127.18.129.3:53179 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:07:14.617183 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.611152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:53179)
0807 15:07:14.611300 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.611337 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.611794 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.611981 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.611984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.612544 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.612730 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.612734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.613335 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.613336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.614099 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.614101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.614401 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.614402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.614496 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.614951 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.614959 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.617035 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
W20260807 15:07:14.617383 32760 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:39963 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.129.62:39963: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
I20260807 15:07:14.620394 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 32545
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:14.621665 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.617597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:53393)
0807 15:07:14.617640 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.617651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.617682 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.617916 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.617917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.618036 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.618084 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.618085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.618672 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.618672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.619204 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.619206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.619508 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.619509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.619568 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.619910 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.619914 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.621595 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 15:07:14.621718 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.617733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:53393 (local address 127.18.129.62:39963)
0807 15:07:14.617774 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.617775 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.617782 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:07:14.617801 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.617801 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.617808 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.617862 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.618150 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.618598 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.619260 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.619430 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.619476 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.621671 (+ 2195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:53393: BlockingRecv error: recv got EOF from 127.18.129.3:53393 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:14 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:14.626271 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.621888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:50631)
0807 15:07:14.621927 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:14.621937 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:14.621975 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:14.622106 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:14.622107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:14.622205 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:14.622229 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.622230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.622983 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.622984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.623738 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:14.623740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.624155 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.624156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:14.624204 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.624530 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:14.624535 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:14.626206 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 15:07:14.626400 32323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:14.621927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:50631 (local address 127.18.129.62:39963)
0807 15:07:14.621956 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.621958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.621965 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:07:14.622027 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.622027 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.622046 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.622103 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.622284 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.622928 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.623807 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.624085 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.624141 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.626288 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:50631: BlockingRecv error: recv got EOF from 127.18.129.3:50631 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260807 15:07:14.626605 32760 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:07:14.628397 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:14.620768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.18.129.62:39963)
0807 15:07:14.620912 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:14.620914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:14.621036 (+ 122us) server_negotiation.cc:407] Connection header received
0807 15:07:14.621101 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:14.621101 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:14.621112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:14.621166 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:14.622066 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.622759 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.623852 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:14.624126 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:14.624181 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:14.625367 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:14.625371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:14.625371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:14.625382 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:14.627125 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:14.627440 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:14.627444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:14.627444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:14.627536 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:14.627719 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:14.627719 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:14.627719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:14.627971 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:14.628119 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:14.628201 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 15:07:14.628302 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
Aug 07 15:07:15 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:15 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:15.379429 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:15.374941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:43451)
0807 15:07:15.375062 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:15.375073 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:15.375126 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:15.375251 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:15.375252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:15.375412 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:15.375463 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:15.375464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.376060 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.376060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:15.376609 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:15.376610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.376924 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.376925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:15.376961 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:15.377401 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:15.377405 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:15.379358 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
W20260807 15:07:15.379509 32766 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:15.374975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:43451 (local address 127.18.129.62:39963)
0807 15:07:15.375095 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:15.375097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:15.375109 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:07:15.375177 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:15.375177 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:15.375187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:15.375241 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:15.375508 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.376008 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.376675 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.376874 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.376919 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:15.379453 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:43451: BlockingRecv error: recv got EOF from 127.18.129.1:43451 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
Aug 07 15:07:15 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:15 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:15.514434 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:15.510257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:38993)
0807 15:07:15.510376 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:15.510387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:15.510441 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:15.510572 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:15.510572 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:15.510735 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:15.510787 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:15.510788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.511474 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.511475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:15.512016 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:15.512018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.512306 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.512306 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:15.512343 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:15.512742 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:15.512747 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:15.514374 (+ 1627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
W20260807 15:07:15.514539 32766 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:15.510335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:38993 (local address 127.18.129.62:39963)
0807 15:07:15.510389 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:15.510391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:15.510440 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:07:15.510501 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:15.510501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:15.510515 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:15.510555 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:15.510836 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.511414 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.512079 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.512258 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.512301 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:15.514467 (+ 2166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:38993: BlockingRecv error: recv got EOF from 127.18.129.2:38993 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 15:07:15 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:15 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:15.631531 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:15.626988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:58625)
0807 15:07:15.627129 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:15.627141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:15.627210 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:15.627333 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:15.627334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:15.627516 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:15.627564 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:15.627566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.628312 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.628313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:15.628884 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:15.628886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.629212 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.629213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:15.629286 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:15.629719 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:15.629724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:15.631460 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
W20260807 15:07:15.631614 32766 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:15.627058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:58625 (local address 127.18.129.62:39963)
0807 15:07:15.627118 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:15.627120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:15.627195 (+ 75us) server_negotiation.cc:407] Connection header received
0807 15:07:15.627271 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:15.627272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:15.627286 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:15.627328 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:15.627637 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.628236 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.628963 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:15.629154 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:15.629206 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:15.631544 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:58625: BlockingRecv error: recv got EOF from 127.18.129.3:58625 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 15:07:16 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:16 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:16.384296 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:16.379915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:58003)
0807 15:07:16.380049 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:16.380059 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:16.380110 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:16.380264 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:16.380265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:16.380432 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:16.380482 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:16.380484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.381126 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.381127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:16.381655 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:16.381656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.381997 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.381998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:16.382048 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:16.382514 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:16.382518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:16.384233 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
W20260807 15:07:16.384419 305 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:16.379996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:58003 (local address 127.18.129.62:39963)
0807 15:07:16.380127 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:16.380129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:16.380136 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:07:16.380158 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:16.380158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:16.380169 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:16.380242 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:16.380548 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.381066 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.381724 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.381929 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.381982 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:16.384343 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:58003: BlockingRecv error: recv got EOF from 127.18.129.1:58003 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Aug 07 15:07:16 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:16 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:16.519461 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:16.514904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:51993)
0807 15:07:16.515033 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:16.515044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:16.515114 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:16.515238 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:16.515239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:16.515404 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:16.515454 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:16.515455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.516093 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.516094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:16.516794 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:16.516797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.517115 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.517115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:16.517154 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:16.517611 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:16.517616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:16.519379 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
W20260807 15:07:16.519547 305 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:16.514967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:51993 (local address 127.18.129.62:39963)
0807 15:07:16.515039 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:16.515041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:16.515107 (+ 66us) server_negotiation.cc:407] Connection header received
0807 15:07:16.515160 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:16.515160 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:16.515170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:16.515220 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:16.515505 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.516018 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.516862 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.517059 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.517110 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:16.519478 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:51993: BlockingRecv error: recv got EOF from 127.18.129.2:51993 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 15:07:16 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:16 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:16.636336 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:16.631956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:58683)
0807 15:07:16.632096 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:16.632107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:16.632162 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:16.632297 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:16.632297 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:16.632462 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:16.632513 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:16.632514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.633131 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.633131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:16.633655 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:16.633657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.633973 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.633974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:16.634009 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:16.634509 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:16.634513 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:16.636276 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
W20260807 15:07:16.636433 305 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:16.632080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:58683 (local address 127.18.129.62:39963)
0807 15:07:16.632137 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:16.632139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:16.632150 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:07:16.632221 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:16.632221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:16.632238 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:16.632285 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:16.632574 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.633076 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.633729 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:16.633915 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:16.633970 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:16.636364 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:58683: BlockingRecv error: recv got EOF from 127.18.129.3:58683 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 15:07:17 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:17 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:17.389221 310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:17.384925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:60617)
0807 15:07:17.385090 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:17.385101 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:17.385162 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:17.385291 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:17.385292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:17.385477 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:17.385530 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:17.385532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.386149 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.386149 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:17.386676 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:17.386677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.386987 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.386988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:17.387033 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:17.387437 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:17.387441 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:17.389158 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
W20260807 15:07:17.389343 311 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:17.384963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:60617 (local address 127.18.129.62:39963)
0807 15:07:17.385114 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:17.385116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:17.385146 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:07:17.385215 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:17.385215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:17.385227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:17.385274 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:17.385587 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.386092 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.386732 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.386936 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.386976 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:17.389261 (+ 2285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:60617: BlockingRecv error: recv got EOF from 127.18.129.1:60617 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
Aug 07 15:07:17 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:17 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:17.524296 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:17.519990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:52287)
0807 15:07:17.520132 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:17.520149 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:17.520224 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:17.520363 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:17.520364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:17.520559 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:17.520612 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:17.520614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.521258 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.521259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:17.521779 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:17.521781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.522109 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.522110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:17.522165 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:17.522566 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:17.522571 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:17.524215 (+ 1644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
W20260807 15:07:17.524376 311 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:17.520106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:52287 (local address 127.18.129.62:39963)
0807 15:07:17.520190 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:17.520192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:17.520204 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:07:17.520292 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:17.520293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:17.520305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:17.520353 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:17.520693 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.521184 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.521857 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.522044 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.522099 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:17.524323 (+ 2224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:52287: BlockingRecv error: recv got EOF from 127.18.129.2:52287 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 15:07:17 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:17 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:17.641263 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:17.636830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:34571)
0807 15:07:17.636987 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:17.636997 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:17.637059 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:17.637185 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:17.637185 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:17.637359 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:17.637412 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:17.637413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.638029 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.638030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:17.638773 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:17.638777 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.639103 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.639104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:17.639141 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:17.639554 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:17.639558 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:17.641200 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
W20260807 15:07:17.641351 311 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:17.636898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:34571 (local address 127.18.129.62:39963)
0807 15:07:17.636964 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:17.636966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:17.637053 (+ 87us) server_negotiation.cc:407] Connection header received
0807 15:07:17.637103 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:17.637104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:17.637118 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:17.637173 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:17.637462 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.637962 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.638837 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:17.639048 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:17.639100 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:17.641288 (+ 2188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:34571: BlockingRecv error: recv got EOF from 127.18.129.3:34571 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 15:07:18 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:18 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:18.394126 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:18.389731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:51871)
0807 15:07:18.389866 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:18.389879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:18.389932 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:18.390092 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:18.390093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:18.390268 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:18.390321 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:18.390323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.390970 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.390971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:18.391510 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:18.391511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.391828 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.391829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:18.391864 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:18.392290 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:18.392294 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:18.394062 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260807 15:07:18.394248 315 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:18.389775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:51871 (local address 127.18.129.62:39963)
0807 15:07:18.389908 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:18.389910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:18.389932 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:07:18.389993 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:18.389994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:18.390008 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:18.390061 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:18.390374 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.390914 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.391582 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.391778 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.391824 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:18.394165 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:51871: BlockingRecv error: recv got EOF from 127.18.129.1:51871 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
Aug 07 15:07:18 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:18 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:18.535069 317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:18.524824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:56085)
0807 15:07:18.524963 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:18.524973 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:18.525032 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:18.525177 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:18.525178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:18.525364 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:18.525416 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:18.525417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.526117 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.526117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:18.526668 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:18.526670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.527120 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.527122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:18.527174 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:18.533156 (+ 5982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:18.533162 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:18.534984 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260807 15:07:18.535158 315 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:18.524912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:56085 (local address 127.18.129.62:39963)
0807 15:07:18.524981 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:18.524983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:18.525024 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:07:18.525091 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:18.525092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:18.525106 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:18.525163 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:18.525539 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.526055 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.526752 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.527030 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.527099 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:18.535097 (+ 7998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:56085: BlockingRecv error: recv got EOF from 127.18.129.2:56085 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 15:07:18 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:18 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:18.646589 319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:18.641810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:60339)
0807 15:07:18.641942 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:18.641955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:18.642030 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:18.642166 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:18.642167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:18.642367 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:18.642475 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:18.642478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.643199 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.643200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:18.643731 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:18.643733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.644063 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.644064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:18.644119 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:18.644630 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:18.644635 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:18.646503 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
W20260807 15:07:18.646711 315 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:18.641894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:60339 (local address 127.18.129.62:39963)
0807 15:07:18.641946 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:18.641949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:18.642023 (+ 74us) server_negotiation.cc:407] Connection header received
0807 15:07:18.642084 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:18.642084 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:18.642101 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:18.642148 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:18.642562 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.643121 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.643809 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:18.644000 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:18.644055 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:18.646630 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:60339: BlockingRecv error: recv got EOF from 127.18.129.3:60339 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 07 15:07:19 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:19 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:19.399411 320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:19.394588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:35307)
0807 15:07:19.394717 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:19.394727 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:19.394784 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:19.394943 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:19.394944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:19.395120 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:19.395172 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:19.395173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.395944 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.395945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:19.396590 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:19.396594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.396958 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.396960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:19.397012 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:19.397554 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:19.397559 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:19.399346 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
W20260807 15:07:19.399514 321 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:19.394623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:35307 (local address 127.18.129.62:39963)
0807 15:07:19.394737 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:19.394740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:19.394769 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:07:19.394831 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:19.394831 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:19.394842 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:19.394933 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:19.395271 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.395856 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.396669 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.396873 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.396930 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:19.399438 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:35307: BlockingRecv error: recv got EOF from 127.18.129.1:35307 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
Aug 07 15:07:19 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:19 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:19.539836 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:19.535552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:55809)
0807 15:07:19.535744 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:19.535755 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:19.535815 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:19.535939 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:19.535940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:19.536115 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:19.536168 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:19.536169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.536773 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.536773 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:19.537335 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:19.537337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.537636 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.537636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:19.537681 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:19.538119 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:19.538123 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:19.539775 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":126,"threads_started":1}
W20260807 15:07:19.539949 321 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:19.535687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:55809 (local address 127.18.129.62:39963)
0807 15:07:19.535757 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:19.535759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:19.535809 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:07:19.535871 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:19.535872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:19.535886 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:19.535925 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:19.536218 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.536716 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.537395 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.537577 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.537621 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:19.539885 (+ 2264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:55809: BlockingRecv error: recv got EOF from 127.18.129.2:55809 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 15:07:19 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:19 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:19.651280 323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:19.647101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:51007)
0807 15:07:19.647251 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:19.647262 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:19.647318 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:19.647430 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:19.647431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:19.647602 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:19.647652 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:19.647653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.648248 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.648249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:19.648769 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:19.648771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.649042 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.649043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:19.649086 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:19.649482 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:19.649485 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:19.651197 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
W20260807 15:07:19.651366 321 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:19.647187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:51007 (local address 127.18.129.62:39963)
0807 15:07:19.647245 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:19.647247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:19.647311 (+ 64us) server_negotiation.cc:407] Connection header received
0807 15:07:19.647360 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:19.647360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:19.647373 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:19.647424 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:19.647698 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.648188 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.648824 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:19.648996 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:19.649041 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:19.651313 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:51007: BlockingRecv error: recv got EOF from 127.18.129.3:51007 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 15:07:20 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:20 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:20.404336 325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:20.399868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:35137)
0807 15:07:20.399999 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:20.400009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:20.400061 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:20.400194 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:20.400195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:20.400388 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:20.400454 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:20.400457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.401164 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.401165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:20.401697 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:20.401700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.402074 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.402076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:20.402159 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:20.402586 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:20.402591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:20.404257 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
W20260807 15:07:20.404429 326 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:20.399920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:35137 (local address 127.18.129.62:39963)
0807 15:07:20.400048 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:20.400050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:20.400058 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:20.400111 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:20.400111 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:20.400123 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:20.400197 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:20.400531 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.401087 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.401772 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.401963 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.402043 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:20.404365 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:35137: BlockingRecv error: recv got EOF from 127.18.129.1:35137 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
Aug 07 15:07:20 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:20 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:20.544982 329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:20.540359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:34999)
0807 15:07:20.540510 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:20.540524 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:20.540609 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:20.540784 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:20.540785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:20.541031 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:20.541097 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:20.541100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.541776 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.541777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:20.542329 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:20.542331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.542630 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.542630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:20.542668 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:20.543123 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:20.543128 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:20.544898 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
W20260807 15:07:20.545056 326 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:20.540472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:34999 (local address 127.18.129.62:39963)
0807 15:07:20.540535 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:20.540537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:20.540601 (+ 64us) server_negotiation.cc:407] Connection header received
0807 15:07:20.540692 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:20.540693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:20.540707 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:20.540767 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:20.541171 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.541685 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.542393 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.542579 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.542622 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:20.545003 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:34999: BlockingRecv error: recv got EOF from 127.18.129.2:34999 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 15:07:20 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:20 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:20.656556 331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:20.651808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:34231)
0807 15:07:20.651970 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:20.651981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:20.652069 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:20.652195 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:20.652196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:20.652372 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:20.652423 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:20.652425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.653072 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.653073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:20.653596 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:20.653597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.653929 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.653929 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:20.653964 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:20.654506 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:20.654509 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:20.656472 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
W20260807 15:07:20.656644 326 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:20.651890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:34231 (local address 127.18.129.62:39963)
0807 15:07:20.651979 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:20.651981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:20.652050 (+ 69us) server_negotiation.cc:407] Connection header received
0807 15:07:20.652124 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:20.652125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:20.652141 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:20.652178 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:20.652496 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.652990 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.653682 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:20.653870 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:20.653926 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:20.656588 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:34231: BlockingRecv error: recv got EOF from 127.18.129.3:34231 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 15:07:21 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:21 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:21.410625 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:21.404935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:37339)
0807 15:07:21.405088 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:21.405105 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:21.405166 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:21.405425 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:21.405427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:21.405620 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:21.405679 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:21.405681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.406393 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.406395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:21.407016 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:21.407019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.407466 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.407467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:21.407524 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:21.408101 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:21.408107 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:21.410536 (+ 2429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
W20260807 15:07:21.410737 335 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:21.405051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:37339 (local address 127.18.129.62:39963)
0807 15:07:21.405202 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:21.405204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:21.405214 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:07:21.405240 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:21.405241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:21.405252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:21.405321 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:21.405773 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.406292 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.407107 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.407319 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.407374 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:21.410661 (+ 3287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:37339: BlockingRecv error: recv got EOF from 127.18.129.1:37339 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
Aug 07 15:07:21 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:21 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:21.549993 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:21.545500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:34247)
0807 15:07:21.545663 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:21.545675 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:21.545737 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:21.545882 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:21.545882 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:21.546054 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:21.546106 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:21.546107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.546761 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.546762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:21.547328 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:21.547330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.547679 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.547680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:21.547725 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:21.548176 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:21.548181 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:21.549916 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
W20260807 15:07:21.550112 335 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:21.545611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:34247 (local address 127.18.129.62:39963)
0807 15:07:21.545686 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:21.545688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:21.545731 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:07:21.545789 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:21.545789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:21.545802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:21.545856 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:21.546169 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.546687 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.547407 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.547607 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.547668 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:21.550022 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:34247: BlockingRecv error: recv got EOF from 127.18.129.2:34247 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 15:07:21 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:21 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:21.662248 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:21.657114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:43001)
0807 15:07:21.657336 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:21.657348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:21.657471 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:21.657653 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:21.657655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:21.657834 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:21.657897 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:21.657898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.658569 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.658570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:21.659136 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:21.659138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.659567 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.659568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:21.659624 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:21.660143 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:21.660148 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:21.662178 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
W20260807 15:07:21.662335 335 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:21.657235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:43001 (local address 127.18.129.62:39963)
0807 15:07:21.657307 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:21.657309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:21.657447 (+ 138us) server_negotiation.cc:407] Connection header received
0807 15:07:21.657523 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:21.657524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:21.657540 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:21.657593 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:21.657982 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.658493 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.659222 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:21.659481 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:21.659538 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:21.662274 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:43001: BlockingRecv error: recv got EOF from 127.18.129.3:43001 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 15:07:22 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:22 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:22.416601 339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:22.411371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:57667)
0807 15:07:22.411505 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:22.411520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:22.411591 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:22.411825 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:22.411826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:22.412039 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:22.412102 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:22.412106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.412974 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.412975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:22.413738 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:22.413742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.414143 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.414145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:22.414202 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:22.414678 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:22.414682 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:22.416470 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
W20260807 15:07:22.416644 342 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:22.411490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:57667 (local address 127.18.129.62:39963)
0807 15:07:22.411671 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:22.411673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:22.411694 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:07:22.411721 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:22.411721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:22.411734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:22.411789 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:22.412197 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.412884 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.413820 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.414083 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.414141 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:22.416578 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:57667: BlockingRecv error: recv got EOF from 127.18.129.1:57667 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":74,"threads_started":1}
Aug 07 15:07:22 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:22 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:22.555531 344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:22.550558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:49147)
0807 15:07:22.550710 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:22.550722 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:22.550804 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:22.550976 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:22.550977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:22.551202 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:22.551257 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:22.551260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.552077 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.552078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:22.552686 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:22.552689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.553009 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.553010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:22.553059 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:22.553594 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:22.553600 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:22.555467 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
W20260807 15:07:22.555630 342 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:22.550634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:49147 (local address 127.18.129.62:39963)
0807 15:07:22.550692 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:22.550694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:22.550795 (+ 101us) server_negotiation.cc:407] Connection header received
0807 15:07:22.550882 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:22.550883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:22.550898 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:22.550951 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:22.551329 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.551996 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.552761 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.552947 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.552990 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:22.555568 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:49147: BlockingRecv error: recv got EOF from 127.18.129.2:49147 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 07 15:07:22 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:22 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:22.667553 345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:22.662783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:33499)
0807 15:07:22.662965 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:22.662978 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:22.663051 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:22.663197 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:22.663198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:22.663402 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:22.663459 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:22.663461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.664218 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.664219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:22.664754 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:22.664756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.665084 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.665086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:22.665142 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:22.665620 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:22.665624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:22.667451 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":109,"threads_started":1}
W20260807 15:07:22.667630 342 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:22.662906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:33499 (local address 127.18.129.62:39963)
0807 15:07:22.662970 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:22.662972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:22.663028 (+ 56us) server_negotiation.cc:407] Connection header received
0807 15:07:22.663116 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:22.663117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:22.663133 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:22.663175 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:22.663558 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.664133 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.664831 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:22.665017 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:22.665078 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:22.667574 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:33499: BlockingRecv error: recv got EOF from 127.18.129.3:33499 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 15:07:23 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:23 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:23.422019 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:23.417136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:52369)
0807 15:07:23.417275 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:23.417291 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:23.417369 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:23.417550 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:23.417551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:23.417782 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:23.417860 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:23.417863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.418554 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.418555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:23.419160 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:23.419163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.419492 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.419493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:23.419537 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:23.419968 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:23.419973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:23.421953 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
W20260807 15:07:23.422111 347 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:23.417222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:52369 (local address 127.18.129.62:39963)
0807 15:07:23.417342 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:23.417345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:23.417370 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:07:23.417453 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:23.417453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:23.417466 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:23.417528 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:23.417945 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.418463 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.419230 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.419427 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.419480 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:23.422036 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:52369: BlockingRecv error: recv got EOF from 127.18.129.1:52369 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
Aug 07 15:07:23 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:23 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:23.564173 349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:23.556136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:52903)
0807 15:07:23.556299 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:23.556310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:23.556374 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:23.556514 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:23.556514 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:23.556697 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:23.556749 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:23.556750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.557399 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.557400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:23.557934 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:23.557935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.558284 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.558284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:23.558321 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:23.562124 (+ 3803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:23.562131 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:23.564107 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
W20260807 15:07:23.564342 347 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:23.556247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:52903 (local address 127.18.129.62:39963)
0807 15:07:23.556307 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:23.556309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:23.556369 (+ 60us) server_negotiation.cc:407] Connection header received
0807 15:07:23.556424 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:23.556424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:23.556438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:23.556498 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:23.556809 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.557317 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.558023 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.558226 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.558279 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:23.564273 (+ 5994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:52903: BlockingRecv error: recv got EOF from 127.18.129.2:52903 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 15:07:23 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:23 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:23.673219 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:23.668147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:58839)
0807 15:07:23.668305 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:23.668317 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:23.668374 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:23.668519 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:23.668520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:23.668732 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:23.668791 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:23.668794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.669578 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.669579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:23.670142 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:23.670145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.670653 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.670654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:23.670722 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:23.671341 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:23.671360 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:23.673145 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
W20260807 15:07:23.673317 347 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:23.668285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:58839 (local address 127.18.129.62:39963)
0807 15:07:23.668343 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:23.668345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:23.668358 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:07:23.668425 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:23.668426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:23.668440 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:23.668505 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:23.668909 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.669479 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.670271 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:23.670553 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:23.670621 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:23.673252 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:58839: BlockingRecv error: recv got EOF from 127.18.129.3:58839 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 15:07:24 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:24 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:24.427083 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:24.422429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:43667)
0807 15:07:24.422590 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:24.422602 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:24.422662 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:24.422933 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:24.422934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:24.423115 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:24.423167 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:24.423169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.423840 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.423841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:24.424379 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:24.424381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.424732 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.424733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:24.424778 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:24.425200 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:24.425204 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:24.427003 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
W20260807 15:07:24.427179 355 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:24.422467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:43667 (local address 127.18.129.62:39963)
0807 15:07:24.422594 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:24.422596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:24.422644 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:07:24.422715 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:24.422715 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:24.422727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:24.422775 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:24.423247 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.423757 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.424454 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.424661 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.424711 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:24.427102 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:43667: BlockingRecv error: recv got EOF from 127.18.129.1:43667 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
Aug 07 15:07:24 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:24 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:24.569119 359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:24.564678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:52739)
0807 15:07:24.564830 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:24.564841 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:24.564894 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:24.565041 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:24.565042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:24.565223 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:24.565275 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:24.565276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.565939 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.565939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:24.566475 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:24.566476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.566784 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.566784 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:24.566821 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:24.567318 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:24.567323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:24.569052 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
W20260807 15:07:24.569216 355 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:24.564799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:52739 (local address 127.18.129.62:39963)
0807 15:07:24.564862 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:24.564864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:24.564879 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:07:24.564948 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:24.564948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:24.564962 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:24.565020 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:24.565360 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.565871 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.566532 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.566733 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.566782 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:24.569154 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:52739: BlockingRecv error: recv got EOF from 127.18.129.2:52739 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 15:07:24 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:24 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:24.677950 360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:24.673673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:45795)
0807 15:07:24.673828 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:24.673840 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:24.673890 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:24.674020 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:24.674021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:24.674204 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:24.674255 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:24.674257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.674870 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.674870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:24.675396 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:24.675397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.675700 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.675700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:24.675739 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:24.676149 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:24.676154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:24.677856 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
W20260807 15:07:24.678018 355 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:24.673801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:45795 (local address 127.18.129.62:39963)
0807 15:07:24.673865 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:24.673866 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:24.673878 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:07:24.673944 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:24.673945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:24.673957 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:24.674006 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:24.674307 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.674796 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.675455 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:24.675644 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:24.675694 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:24.677962 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:45795: BlockingRecv error: recv got EOF from 127.18.129.3:45795 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 15:07:25 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:25 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:25.432089 361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:25.427479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:49069)
0807 15:07:25.427623 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:25.427636 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:25.427702 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:25.427864 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:25.427864 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:25.428032 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:25.428085 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:25.428087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.428763 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.428764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:25.429351 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:25.429353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.429689 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.429690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:25.429730 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:25.430213 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:25.430217 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:25.432013 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
W20260807 15:07:25.432183 362 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:25.427542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:49069 (local address 127.18.129.62:39963)
0807 15:07:25.427720 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:25.427722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:25.427733 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:07:25.427758 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:25.427759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:25.427788 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:25.427859 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:25.428148 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.428695 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.429420 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.429635 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.429685 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:25.432107 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:49069: BlockingRecv error: recv got EOF from 127.18.129.1:49069 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":117,"threads_started":1}
Aug 07 15:07:25 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:25 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:25.574031 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:25.569615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:43705)
0807 15:07:25.569758 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:25.569772 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:25.569847 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:25.569973 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:25.569974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:25.570183 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:25.570231 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:25.570233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.570947 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.570948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:25.571506 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:25.571509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.571893 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.571894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:25.571936 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:25.572356 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:25.572360 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:25.573958 (+ 1598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260807 15:07:25.574107 362 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:25.569714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:43705 (local address 127.18.129.62:39963)
0807 15:07:25.569779 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:25.569781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:25.569825 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:07:25.569900 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:25.569901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:25.569914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:25.569961 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:25.570305 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.570839 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.571617 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.571824 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.571898 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:25.574052 (+ 2154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:43705: BlockingRecv error: recv got EOF from 127.18.129.2:43705 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 15:07:25 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:25 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:25.683202 367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:25.678458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:55227)
0807 15:07:25.678612 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:25.678626 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:25.678698 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:25.678831 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:25.678832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:25.679067 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:25.679120 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:25.679122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.679796 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.679797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:25.680350 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:25.680352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.680691 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.680692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:25.680748 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:25.681177 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:25.681182 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:25.683129 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
W20260807 15:07:25.683279 362 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:25.678547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:55227 (local address 127.18.129.62:39963)
0807 15:07:25.678626 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:25.678628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:25.678677 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:07:25.678749 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:25.678750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:25.678766 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:25.678816 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:25.679202 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.679715 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.680432 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:25.680630 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:25.680685 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:25.683223 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:55227: BlockingRecv error: recv got EOF from 127.18.129.3:55227 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 15:07:26 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:26 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:26.437314 368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:26.432499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:35321)
0807 15:07:26.432650 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:26.432661 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:26.432710 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:26.432835 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:26.432836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:26.433002 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:26.433080 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:26.433082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.433709 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.433709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:26.434246 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:26.434248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.434674 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.434675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:26.434710 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:26.435194 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:26.435198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:26.437228 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
W20260807 15:07:26.437480 369 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:26.432563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:35321 (local address 127.18.129.62:39963)
0807 15:07:26.432682 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:26.432684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:26.432695 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:07:26.432754 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:26.432755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:26.432765 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:26.432829 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:26.433133 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.433649 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.434395 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.434603 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.434678 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:26.437343 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:35321: BlockingRecv error: recv got EOF from 127.18.129.1:35321 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
Aug 07 15:07:26 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:26 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:26.579041 370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:26.574532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:60991)
0807 15:07:26.574707 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:26.574718 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:26.574770 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:26.574926 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:26.574926 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:26.575126 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:26.575176 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:26.575178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.575805 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.575805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:26.576372 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:26.576374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.576789 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.576790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:26.576854 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:26.577289 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:26.577294 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:26.578978 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
W20260807 15:07:26.579150 369 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:26.574684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:60991 (local address 127.18.129.62:39963)
0807 15:07:26.574741 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:26.574743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:26.574759 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:07:26.574825 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:26.574826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:26.574837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:26.574909 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:26.575231 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.575745 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.576470 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.576701 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.576760 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:26.579090 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:60991: BlockingRecv error: recv got EOF from 127.18.129.2:60991 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 15:07:26 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:26 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:26.688089 371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:26.683690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:45941)
0807 15:07:26.683834 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:26.683847 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:26.683914 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:26.684045 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:26.684046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:26.684224 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:26.684275 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:26.684276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.684944 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.684945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:26.685492 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:26.685494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.685802 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.685803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:26.685849 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:26.686254 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:26.686258 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:26.688000 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
W20260807 15:07:26.688158 369 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:26.683720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:45941 (local address 127.18.129.62:39963)
0807 15:07:26.683786 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:26.683788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:26.683894 (+ 106us) server_negotiation.cc:407] Connection header received
0807 15:07:26.683966 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:26.683967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:26.683983 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:26.684031 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:26.684329 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.684876 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.685553 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:26.685744 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:26.685787 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:26.688103 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:45941: BlockingRecv error: recv got EOF from 127.18.129.3:45941 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 15:07:27 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:27 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:27.442265 372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:27.437893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:56935)
0807 15:07:27.438043 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:27.438055 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:27.438122 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:27.438259 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:27.438260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:27.438432 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:27.438482 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:27.438484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.439168 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.439168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:27.439705 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:27.439707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.440004 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.440004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:27.440042 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:27.440489 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:27.440494 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:27.442180 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
W20260807 15:07:27.442346 374 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:27.437972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:56935 (local address 127.18.129.62:39963)
0807 15:07:27.438090 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:27.438092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:27.438104 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:07:27.438179 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:27.438179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:27.438191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:27.438239 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:27.438548 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.439106 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.439765 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.439952 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.439996 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:27.442280 (+ 2284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:56935: BlockingRecv error: recv got EOF from 127.18.129.1:56935 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":75,"threads_started":1}
Aug 07 15:07:27 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:27 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:27.584367 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:27.579643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:60123)
0807 15:07:27.579799 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:27.579822 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:27.579894 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:27.580047 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:27.580047 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:27.580238 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:27.580291 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:27.580293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.580943 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.580944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:27.581505 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:27.581507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.581806 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.581807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:27.581851 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:27.582366 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:27.582372 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:27.584302 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
W20260807 15:07:27.584614 374 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:27.579834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:60123 (local address 127.18.129.62:39963)
0807 15:07:27.579894 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:27.579896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:27.579906 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:07:27.579945 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:27.579945 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:27.579958 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:27.580007 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:27.580352 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.580867 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.581581 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.581755 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.581796 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:27.584554 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:60123: BlockingRecv error: recv got EOF from 127.18.129.2:60123 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 15:07:27 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:27 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:27.693109 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:27.688561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:50847)
0807 15:07:27.688739 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:27.688750 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:27.688816 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:27.688958 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:27.688959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:27.689140 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:27.689190 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:27.689191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.689852 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.689853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:27.690414 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:27.690417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.690738 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.690739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:27.690794 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:27.691269 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:27.691274 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:27.693032 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":99,"threads_started":1}
W20260807 15:07:27.693197 374 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:27.688634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:50847 (local address 127.18.129.62:39963)
0807 15:07:27.688695 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:27.688698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:27.688808 (+ 110us) server_negotiation.cc:407] Connection header received
0807 15:07:27.688869 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:27.688870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:27.688894 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:27.688941 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:27.689261 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.689773 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.690496 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:27.690679 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:27.690733 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:27.693135 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:50847: BlockingRecv error: recv got EOF from 127.18.129.3:50847 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 15:07:28 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:28 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:28.447418 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:28.442763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:49489)
0807 15:07:28.442919 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:28.442932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:28.442992 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:28.443136 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:28.443137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:28.443307 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:28.443359 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:28.443361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.443964 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.443965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:28.444714 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:28.444718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.445036 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.445037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:28.445075 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:28.445539 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:28.445544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:28.447345 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
W20260807 15:07:28.447503 379 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:28.442892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:49489 (local address 127.18.129.62:39963)
0807 15:07:28.443000 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:28.443002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:28.443010 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:28.443056 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:28.443056 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:28.443067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:28.443125 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:28.443417 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.443910 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.444780 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.444982 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.445040 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:28.447440 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:49489: BlockingRecv error: recv got EOF from 127.18.129.1:49489 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
Aug 07 15:07:28 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:28 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:28.594058 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:28.584883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:51193)
0807 15:07:28.585076 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:28.585088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:28.585159 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:28.585321 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:28.585322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:28.585551 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:28.585610 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:28.585612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.586255 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.586256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:28.586770 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:28.586771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.587128 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.587128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:28.587165 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:28.592302 (+ 5137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:28.592307 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:28.593994 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
W20260807 15:07:28.594174 379 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:28.584960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:51193 (local address 127.18.129.62:39963)
0807 15:07:28.585044 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:28.585047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:28.585149 (+ 102us) server_negotiation.cc:407] Connection header received
0807 15:07:28.585231 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:28.585232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:28.585248 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:28.585302 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:28.585683 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.586193 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.586839 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.587073 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.587127 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:28.594108 (+ 6981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:51193: BlockingRecv error: recv got EOF from 127.18.129.2:51193 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 15:07:28 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:28 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.3@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:28.698167 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:28.693572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.3:42685)
0807 15:07:28.693740 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:28.693753 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:28.693823 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:28.693955 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:28.693956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:28.694152 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:28.694202 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:28.694204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.694902 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.694903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:28.695452 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:28.695454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.695830 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.695831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:28.695883 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:28.696305 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:28.696310 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:28.698072 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
W20260807 15:07:28.698236 379 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:28.693676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:42685 (local address 127.18.129.62:39963)
0807 15:07:28.693747 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:28.693749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:28.693803 (+ 54us) server_negotiation.cc:407] Connection header received
0807 15:07:28.693875 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:28.693876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:28.693891 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:28.693941 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:28.694278 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.694783 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.695539 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:28.695729 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:28.695787 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:28.698181 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.3:42685: BlockingRecv error: recv got EOF from 127.18.129.3:42685 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 15:07:29 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:29 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.1@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:29.452225 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:29.447907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.1:47295)
0807 15:07:29.448049 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:29.448060 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:29.448112 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:29.448225 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:29.448225 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:29.448388 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:29.448443 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:29.448445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.449050 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.449051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:29.449585 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:29.449586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.449882 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.449883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:29.449920 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:29.450334 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:29.450338 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:29.452157 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
W20260807 15:07:29.452329 386 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:29.447979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:47295 (local address 127.18.129.62:39963)
0807 15:07:29.448117 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:29.448119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:29.448130 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:07:29.448154 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:29.448155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:29.448165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:29.448214 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:29.448491 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.448995 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.449644 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.449832 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.449878 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:29.452253 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.1:47295: BlockingRecv error: recv got EOF from 127.18.129.1:47295 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
Aug 07 15:07:29 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:07:29 dist-test-slave-5bn2 krb5kdc[31164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.129.2@KRBTEST.COM for oryx/127.18.129.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:07:29.599741 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:29.594692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39963 (local address 127.18.129.2:32801)
0807 15:07:29.594840 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:29.594879 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:29.594948 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:29.595105 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:29.595105 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:29.595275 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:29.595330 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:29.595331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.596266 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.596267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:29.596862 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:29.596864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.597252 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.597253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:29.597298 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:29.597785 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:29.597791 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:29.599659 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.129.62:39963: Server oryx/127.18.129.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
W20260807 15:07:29.599820 386 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:29.594781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:32801 (local address 127.18.129.62:39963)
0807 15:07:29.594836 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:29.594838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:29.594939 (+ 101us) server_negotiation.cc:407] Connection header received
0807 15:07:29.594999 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:29.595000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:29.595012 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:29.595078 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:29.595387 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.596138 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.596961 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.597186 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.597242 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:29.599767 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.129.2:32801: BlockingRecv error: recv got EOF from 127.18.129.2:32801 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:07:29.627629 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 32090
I20260807 15:07:29.639873 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 32324
I20260807 15:07:29.649355 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 32545
I20260807 15:07:29.661322 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 31969
2026-08-07T15:07:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17574 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T15:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:29Z Disabled control of system clock
I20260807 15:07:29.697335 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43549
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:43549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:07:29.777773 398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:29.777987 398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:29.778005 398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:29.779162 398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:29.779197 398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:29.779208 398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:29.779219 398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:29.779229 398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:29.780527 398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:43549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:29.780728 398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:29.780964 398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:29.785737 413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:29.785723 417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:29.785754 412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:29.785866 398 server_base.cc:1034] running on GCE node
I20260807 15:07:29.786289 398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:29.786563 398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:29.787707 398 hybrid_clock.cc:648] HybridClock initialized: now 1786115249787699 us; error 28 us; skew 500 ppm
I20260807 15:07:29.788880 398 webserver.cc:460] Webserver started at http://127.18.129.62:44023/ using document root <none> and password file <none>
I20260807 15:07:29.789110 398 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:29.789160 398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:29.789269 398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:29.790360 398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "c3300e97c50b40159fc52dc9442bf329"
format_stamp: "Formatted at 2026-08-07 15:07:29 on dist-test-slave-5bn2"
server_key: "959e2b7f2ae602d271bbc300ffde9d48"
server_key_iv: "431970b49ec216b9e5a4cee010c59137"
server_key_version: "encryptionkey@0"
I20260807 15:07:29.790716 398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "c3300e97c50b40159fc52dc9442bf329"
format_stamp: "Formatted at 2026-08-07 15:07:29 on dist-test-slave-5bn2"
server_key: "959e2b7f2ae602d271bbc300ffde9d48"
server_key_iv: "431970b49ec216b9e5a4cee010c59137"
server_key_version: "encryptionkey@0"
I20260807 15:07:29.791997 398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:07:29.792696 423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:29.792850 398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:07:29.792948 398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "c3300e97c50b40159fc52dc9442bf329"
format_stamp: "Formatted at 2026-08-07 15:07:29 on dist-test-slave-5bn2"
server_key: "959e2b7f2ae602d271bbc300ffde9d48"
server_key_iv: "431970b49ec216b9e5a4cee010c59137"
server_key_version: "encryptionkey@0"
I20260807 15:07:29.793012 398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:29.814510 398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:29.814819 398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:29.814992 398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:29.818867 398 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:43549
I20260807 15:07:29.818899 500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:43549 every 8 connection(s)
I20260807 15:07:29.819226 398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:29.819882 503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:29.821739 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 398
I20260807 15:07:29.821852 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:29.822044 18948 external_mini_cluster.cc:1428] Setting key bfb4015500cc28f85b91e92ad5f4b762
I20260807 15:07:29.823024 503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329: Bootstrap starting.
I20260807 15:07:29.823711 503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:29.823985 503 log.cc:824] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:29.824677 503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329: No bootstrap required, opened a new log
I20260807 15:07:29.826902 503 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3300e97c50b40159fc52dc9442bf329" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43549 } }
I20260807 15:07:29.827069 503 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:29.827095 503 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3300e97c50b40159fc52dc9442bf329, State: Initialized, Role: FOLLOWER
I20260807 15:07:29.827210 503 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3300e97c50b40159fc52dc9442bf329" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43549 } }
I20260807 15:07:29.827301 503 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:29.827333 503 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:29.827405 503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:29.827700 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:29.822576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.18.129.62:43549)
0807 15:07:29.822749 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:29.822756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:29.822781 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:07:29.823220 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:29.823227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:29.823290 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:29.823401 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:29.823636 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.824667 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.825729 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.826149 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.826231 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:29.826936 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:29.826940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:29.826940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:29.827130 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:29.827171 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:29.827186 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:29.827249 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:29.827322 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:07:29.827497 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260807 15:07:29.828012 503 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3300e97c50b40159fc52dc9442bf329" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43549 } }
I20260807 15:07:29.828140 503 leader_election.cc:302] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3300e97c50b40159fc52dc9442bf329; no voters:
I20260807 15:07:29.828305 503 leader_election.cc:288] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:29.828351 511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:29.828538 511 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [term 1 LEADER]: Becoming Leader. State: Replica: c3300e97c50b40159fc52dc9442bf329, State: Running, Role: LEADER
I20260807 15:07:29.828653 503 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:29.828679 511 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3300e97c50b40159fc52dc9442bf329" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43549 } }
I20260807 15:07:29.829048 516 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3300e97c50b40159fc52dc9442bf329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3300e97c50b40159fc52dc9442bf329" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43549 } } }
I20260807 15:07:29.829061 517 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3300e97c50b40159fc52dc9442bf329. Latest consensus state: current_term: 1 leader_uuid: "c3300e97c50b40159fc52dc9442bf329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3300e97c50b40159fc52dc9442bf329" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43549 } } }
I20260807 15:07:29.829281 517 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:29.829191 516 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:29.829727 527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:29.830262 527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:29.831774 527 catalog_manager.cc:1347] Generated new cluster ID: 14daacaea3914725b2d2587f0e556f53
I20260807 15:07:29.831820 527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:29.841735 527 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:29.842434 527 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:29.855486 527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3300e97c50b40159fc52dc9442bf329: Generated new TSK 0
I20260807 15:07:29.855695 527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:07:29.861773 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43549
--builtin_ntp_servers=127.18.129.20:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:07:29.947713 542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:29.947929 542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:29.947952 542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:29.949137 542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:29.949182 542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:29.949205 542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:29.950627 542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:29.950896 542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:29.951207 542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:29.957290 552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:29.957298 558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:29.957298 553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:29.957434 542 server_base.cc:1034] running on GCE node
I20260807 15:07:29.957743 542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:29.957975 542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:29.959115 542 hybrid_clock.cc:648] HybridClock initialized: now 1786115249959106 us; error 33 us; skew 500 ppm
I20260807 15:07:29.960382 542 webserver.cc:460] Webserver started at http://127.18.129.1:33029/ using document root <none> and password file <none>
I20260807 15:07:29.960649 542 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:29.960704 542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:29.960827 542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:29.961925 542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "4942e19aebbb4c57ae26882363f8ccdb"
format_stamp: "Formatted at 2026-08-07 15:07:29 on dist-test-slave-5bn2"
server_key: "bf29dfd4fc6201072a30278c54563275"
server_key_iv: "006db3b591981798f06d28826f9e7bc5"
server_key_version: "encryptionkey@0"
I20260807 15:07:29.962255 542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "4942e19aebbb4c57ae26882363f8ccdb"
format_stamp: "Formatted at 2026-08-07 15:07:29 on dist-test-slave-5bn2"
server_key: "bf29dfd4fc6201072a30278c54563275"
server_key_iv: "006db3b591981798f06d28826f9e7bc5"
server_key_version: "encryptionkey@0"
I20260807 15:07:29.963577 542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:29.964301 565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:29.964455 542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:29.964563 542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "4942e19aebbb4c57ae26882363f8ccdb"
format_stamp: "Formatted at 2026-08-07 15:07:29 on dist-test-slave-5bn2"
server_key: "bf29dfd4fc6201072a30278c54563275"
server_key_iv: "006db3b591981798f06d28826f9e7bc5"
server_key_version: "encryptionkey@0"
I20260807 15:07:29.964629 542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:29.982842 542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:29.983175 542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:29.983305 542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:29.983503 542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:29.983829 542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:29.983862 542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:29.983892 542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:29.983909 542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:29.989485 542 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:42997
I20260807 15:07:29.989545 725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:42997 every 8 connection(s)
I20260807 15:07:29.989861 542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:07:29.994843 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:29.990627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:46847 (local address 127.18.129.62:43549)
0807 15:07:29.990678 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:29.990680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:29.990891 (+ 211us) server_negotiation.cc:407] Connection header received
0807 15:07:29.991420 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:29.991421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:29.991434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:29.991486 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:29.992135 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.992629 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.993587 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.993773 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.993828 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:29.994588 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:29.994590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:29.994590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:29.994595 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:29.994612 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:29.994616 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:29.994659 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:29.994734 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 15:07:29.994761 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:07:29.995083 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:29.990553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43549 (local address 127.18.129.1:46847)
0807 15:07:29.990768 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:29.990803 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:29.991325 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:29.991557 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:29.991562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:29.991572 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:29.992050 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:29.992055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.992713 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.992714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:29.993501 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:29.993504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:29.993843 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:29.993844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:29.993973 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:29.994460 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:29.994461 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:29.994489 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:29.994512 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:29.994664 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:29.994670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:29.994673 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:29.994734 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:07:29.994838 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260807 15:07:29.995651 727 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43549
I20260807 15:07:29.995745 727 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:29.995918 727 heartbeater.cc:511] Master 127.18.129.62:43549 requested a full tablet report, sending...
I20260807 15:07:29.996415 451 ts_manager.cc:194] Registered new tserver with Master: 4942e19aebbb4c57ae26882363f8ccdb (127.18.129.1:42997)
I20260807 15:07:29.997023 451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.1:46847
I20260807 15:07:29.997493 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 542
I20260807 15:07:29.997587 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:07:29.997761 18948 external_mini_cluster.cc:1428] Setting key 9503f5fed6482b2d001a0da67e7c185f
I20260807 15:07:29.998541 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43549
--builtin_ntp_servers=127.18.129.20:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:07:30.080206 734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:30.080416 734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:30.080436 734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:30.081544 734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:30.081578 734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:30.081600 734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:30.082913 734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:30.083117 734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:30.083418 734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:30.088454 743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.088570 742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:30.088569 734 server_base.cc:1034] running on GCE node
W20260807 15:07:30.088462 747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:30.088902 734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:30.089155 734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:30.090299 734 hybrid_clock.cc:648] HybridClock initialized: now 1786115250090284 us; error 29 us; skew 500 ppm
I20260807 15:07:30.091548 734 webserver.cc:460] Webserver started at http://127.18.129.2:44943/ using document root <none> and password file <none>
I20260807 15:07:30.091794 734 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:30.091846 734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:30.091974 734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:30.093053 734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "6cb6839a7f0748c68870e6dbd37440be"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "79de50cc82b8a97395a98b047563f829"
server_key_iv: "a123716e39a445b887bcd721fb91a106"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.093405 734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "6cb6839a7f0748c68870e6dbd37440be"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "79de50cc82b8a97395a98b047563f829"
server_key_iv: "a123716e39a445b887bcd721fb91a106"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.094661 734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:07:30.095420 758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.095599 734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.095685 734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "6cb6839a7f0748c68870e6dbd37440be"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "79de50cc82b8a97395a98b047563f829"
server_key_iv: "a123716e39a445b887bcd721fb91a106"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.095764 734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:30.107326 734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:30.107625 734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:30.107750 734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:30.107977 734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:30.108294 734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:30.108327 734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.108356 734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:30.108379 734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.114133 734 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:33071
I20260807 15:07:30.114192 949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:33071 every 8 connection(s)
I20260807 15:07:30.114508 734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:07:30.119357 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.115237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:35941 (local address 127.18.129.62:43549)
0807 15:07:30.115303 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:30.115305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:30.115527 (+ 222us) server_negotiation.cc:407] Connection header received
0807 15:07:30.116000 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:30.116001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:30.116012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:30.116064 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:30.116689 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:30.117173 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:30.118113 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:30.118305 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:30.118358 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:30.119140 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:30.119141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:30.119141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:30.119146 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:30.119161 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:30.119165 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:30.119210 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:30.119272 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:07:30.119295 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:07:30.119558 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.115219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43549 (local address 127.18.129.2:35941)
0807 15:07:30.115410 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:30.115446 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:30.115927 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:30.116132 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:30.116135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:30.116143 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:30.116627 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:30.116632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:30.117237 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:30.117237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:30.118031 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:30.118033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:30.118365 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:30.118366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:30.118470 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:30.118992 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:30.118992 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:30.119023 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:30.119048 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:30.119216 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:30.119219 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:30.119221 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:07:30.119267 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:07:30.119350 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260807 15:07:30.120009 952 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43549
I20260807 15:07:30.120105 952 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:30.120291 952 heartbeater.cc:511] Master 127.18.129.62:43549 requested a full tablet report, sending...
I20260807 15:07:30.120654 451 ts_manager.cc:194] Registered new tserver with Master: 6cb6839a7f0748c68870e6dbd37440be (127.18.129.2:33071)
I20260807 15:07:30.120992 451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.2:35941
I20260807 15:07:30.123966 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 734
I20260807 15:07:30.124046 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:07:30.124215 18948 external_mini_cluster.cc:1428] Setting key 53f47ae6a8928359bf83a12e5f49d203
I20260807 15:07:30.125047 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43549
--builtin_ntp_servers=127.18.129.20:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:07:30.208150 955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:30.208366 955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:30.208396 955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:30.209545 955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:30.209587 955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:30.209619 955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:30.211022 955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:30.211258 955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:30.211567 955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:30.216560 960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.216575 964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.216601 968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:30.216903 955 server_base.cc:1034] running on GCE node
I20260807 15:07:30.217180 955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:30.217414 955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:30.218554 955 hybrid_clock.cc:648] HybridClock initialized: now 1786115250218534 us; error 32 us; skew 500 ppm
I20260807 15:07:30.219857 955 webserver.cc:460] Webserver started at http://127.18.129.3:39509/ using document root <none> and password file <none>
I20260807 15:07:30.220106 955 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:30.220151 955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:30.220249 955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:30.221312 955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "f70895343ab1435b9fae24d0ad28ce73"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "1d842834b2ddb3c70eb0b184a4df8b17"
server_key_iv: "d9344709096d63dcc9f54eede7af0cd2"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.221627 955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "f70895343ab1435b9fae24d0ad28ce73"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "1d842834b2ddb3c70eb0b184a4df8b17"
server_key_iv: "d9344709096d63dcc9f54eede7af0cd2"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.222965 955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:07:30.223659 983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.223798 955 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:30.223869 955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "f70895343ab1435b9fae24d0ad28ce73"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "1d842834b2ddb3c70eb0b184a4df8b17"
server_key_iv: "d9344709096d63dcc9f54eede7af0cd2"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.223941 955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:30.231513 955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:30.231782 955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:30.231902 955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:30.232110 955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:30.232419 955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:30.232453 955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.232481 955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:30.232506 955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.237952 955 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:38257
I20260807 15:07:30.238016 1152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:38257 every 8 connection(s)
I20260807 15:07:30.238313 955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:07:30.240289 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 955
I20260807 15:07:30.240394 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:07:30.240595 18948 external_mini_cluster.cc:1428] Setting key 37ae021e98f799ed249a9bae8ef5a13d
I20260807 15:07:30.243985 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.239047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:49215 (local address 127.18.129.62:43549)
0807 15:07:30.239098 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:30.239100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:30.239373 (+ 273us) server_negotiation.cc:407] Connection header received
0807 15:07:30.239876 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:30.239877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:30.239888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:30.239940 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:30.240796 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:30.241531 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:30.242679 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:30.242928 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:30.242983 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:30.243726 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:30.243727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:30.243728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:30.243732 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:30.243748 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:30.243752 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:30.243800 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:30.243876 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 15:07:30.243904 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:07:30.244185 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.239069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43549 (local address 127.18.129.3:49215)
0807 15:07:30.239257 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:30.239291 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:30.239794 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:30.240003 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:30.240007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:30.240016 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:30.240710 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:30.240717 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:30.241632 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:30.241633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:30.242595 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:30.242599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:30.243012 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:30.243013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:30.243147 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:30.243600 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:30.243600 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:30.243627 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:30.243650 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:30.243805 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:30.243810 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:30.243813 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:30.243874 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:07:30.243979 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260807 15:07:30.244609 1153 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43549
I20260807 15:07:30.244700 1153 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:30.244850 1153 heartbeater.cc:511] Master 127.18.129.62:43549 requested a full tablet report, sending...
I20260807 15:07:30.245205 451 ts_manager.cc:194] Registered new tserver with Master: f70895343ab1435b9fae24d0ad28ce73 (127.18.129.3:38257)
I20260807 15:07:30.245530 451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.3:49215
I20260807 15:07:30.253436 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:07:30.255533 508 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:30.254968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.18.129.62:43549)
0807 15:07:30.255020 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:30.255022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:30.255173 (+ 151us) server_negotiation.cc:407] Connection header received
0807 15:07:30.255243 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:30.255244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:30.255255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:30.255302 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:30.255450 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56354: BlockingRecv error: recv got EOF from 127.0.0.1:56354 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:07:30.255612 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 542
I20260807 15:07:30.273058 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 734
I20260807 15:07:30.285528 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 955
I20260807 15:07:30.296077 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 398
2026-08-07T15:07:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (634 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T15:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:30Z Disabled control of system clock
I20260807 15:07:30.331810 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:44701
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:07:30.413108 1165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:30.413323 1165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:30.413344 1165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:30.414518 1165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:30.414566 1165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:30.414582 1165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:30.414594 1165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:30.414618 1165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:30.416154 1165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:44701
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:30.416414 1165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:30.416700 1165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:30.421701 1183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.421772 1176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.421711 1177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:30.421934 1165 server_base.cc:1034] running on GCE node
I20260807 15:07:30.422241 1165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:30.422539 1165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:30.423755 1165 hybrid_clock.cc:648] HybridClock initialized: now 1786115250423738 us; error 45 us; skew 500 ppm
I20260807 15:07:30.424978 1165 webserver.cc:460] Webserver started at http://127.18.129.62:44175/ using document root <none> and password file <none>
I20260807 15:07:30.425185 1165 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:30.425227 1165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:30.425324 1165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:30.426393 1165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "22fc6e6430db4efb9405e308dcf60923"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "d52d0b0aeef7a87bea01e771baaaf9ca"
server_key_iv: "81eb276106347c12bc838d88d694f276"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.426697 1165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "22fc6e6430db4efb9405e308dcf60923"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "d52d0b0aeef7a87bea01e771baaaf9ca"
server_key_iv: "81eb276106347c12bc838d88d694f276"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.427937 1165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:30.428618 1192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.428772 1165 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:30.428862 1165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "22fc6e6430db4efb9405e308dcf60923"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "d52d0b0aeef7a87bea01e771baaaf9ca"
server_key_iv: "81eb276106347c12bc838d88d694f276"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.428922 1165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:30.432265 1165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:30.432524 1165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:30.432642 1165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:30.436554 1165 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:44701
I20260807 15:07:30.436589 1263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:44701 every 8 connection(s)
I20260807 15:07:30.436898 1165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:30.437549 1264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:30.440941 1264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923: Bootstrap starting.
I20260807 15:07:30.441483 1264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:30.441728 1264 log.cc:824] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:30.442324 1264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923: No bootstrap required, opened a new log
I20260807 15:07:30.443954 1264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fc6e6430db4efb9405e308dcf60923" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 44701 } }
I20260807 15:07:30.444099 1264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:30.444123 1264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fc6e6430db4efb9405e308dcf60923, State: Initialized, Role: FOLLOWER
I20260807 15:07:30.444223 1264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fc6e6430db4efb9405e308dcf60923" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 44701 } }
I20260807 15:07:30.444278 1264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:30.444306 1264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:30.444346 1264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:30.444851 1264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fc6e6430db4efb9405e308dcf60923" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 44701 } }
I20260807 15:07:30.444947 1264 leader_election.cc:302] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22fc6e6430db4efb9405e308dcf60923; no voters:
I20260807 15:07:30.445098 1264 leader_election.cc:288] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:30.445166 1270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:30.445335 1264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:30.445426 1270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [term 1 LEADER]: Becoming Leader. State: Replica: 22fc6e6430db4efb9405e308dcf60923, State: Running, Role: LEADER
I20260807 15:07:30.445533 1270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fc6e6430db4efb9405e308dcf60923" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 44701 } }
I20260807 15:07:30.446161 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 1165
I20260807 15:07:30.446172 1275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22fc6e6430db4efb9405e308dcf60923. Latest consensus state: current_term: 1 leader_uuid: "22fc6e6430db4efb9405e308dcf60923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fc6e6430db4efb9405e308dcf60923" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 44701 } } }
I20260807 15:07:30.446254 1275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:30.446266 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:30.446481 18948 external_mini_cluster.cc:1428] Setting key ff072120c4dd8251c02bcd5b9080d3e0
I20260807 15:07:30.446570 1273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22fc6e6430db4efb9405e308dcf60923" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fc6e6430db4efb9405e308dcf60923" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 44701 } } }
I20260807 15:07:30.446637 1273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923 [sys.catalog]: This master's current role is: LEADER
W20260807 15:07:30.447299 1294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:07:30.447356 1294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:07:30.447541 1283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:30.447710 1283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:30.448973 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.447174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.18.129.62:44701)
0807 15:07:30.447380 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:30.447389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:30.447432 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:07:30.447537 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:30.447542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:30.447590 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:30.447735 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:30.448471 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:30.448475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:30.448475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:30.448480 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:30.448537 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:30.448553 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:30.448649 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:30.448727 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:07:30.448785 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260807 15:07:30.449306 1283 catalog_manager.cc:1347] Generated new cluster ID: 3c606acbd0e1464eaf083ff6e4e266cf
I20260807 15:07:30.449362 1283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:07:30.458665 1283 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:30.458833 1283 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:30.464915 1283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22fc6e6430db4efb9405e308dcf60923: Generated new TSK 0
I20260807 15:07:30.465093 1283 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:07:30.472808 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:44701
--builtin_ntp_servers=127.18.129.20:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:07:30.555394 1303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:30.555608 1303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:30.555629 1303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:30.556761 1303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:30.556802 1303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:30.556825 1303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:30.558259 1303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:30.558488 1303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:30.558766 1303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:30.564730 1312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.564767 1315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:30.564893 1303 server_base.cc:1034] running on GCE node
W20260807 15:07:30.564764 1318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:30.565254 1303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:30.565543 1303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:30.566711 1303 hybrid_clock.cc:648] HybridClock initialized: now 1786115250566693 us; error 34 us; skew 500 ppm
I20260807 15:07:30.568181 1303 webserver.cc:460] Webserver started at http://127.18.129.1:38177/ using document root <none> and password file <none>
I20260807 15:07:30.568427 1303 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:30.568481 1303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:30.568612 1303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:30.569975 1303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "be9632e98b50469183e19f9703af9c32"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "c49d2846fead6b0b927ecb84e673cb74"
server_key_iv: "0dff30616d3bd730f372899b14d7fe1f"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.570392 1303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "be9632e98b50469183e19f9703af9c32"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "c49d2846fead6b0b927ecb84e673cb74"
server_key_iv: "0dff30616d3bd730f372899b14d7fe1f"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.572160 1303 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 15:07:30.573024 1327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.573220 1303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:07:30.573315 1303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "be9632e98b50469183e19f9703af9c32"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "c49d2846fead6b0b927ecb84e673cb74"
server_key_iv: "0dff30616d3bd730f372899b14d7fe1f"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.573387 1303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:30.577140 1303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:30.577364 1303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:30.577461 1303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:30.577625 1303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:30.577864 1303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:30.577894 1303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.577913 1303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:30.577929 1303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.583316 1303 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:41265
I20260807 15:07:30.583379 1520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:41265 every 8 connection(s)
I20260807 15:07:30.583681 1303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:07:30.586434 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.584366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:56109 (local address 127.18.129.62:44701)
0807 15:07:30.584418 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:30.584420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:30.584641 (+ 221us) server_negotiation.cc:407] Connection header received
0807 15:07:30.585077 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:30.585078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:30.585085 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:30.585137 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:30.586247 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:30.586248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:30.586248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:30.586251 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:30.586267 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:30.586272 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:30.586316 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:30.586366 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 15:07:30.586379 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:07:30.586606 1523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.584351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:44701 (local address 127.18.129.1:56109)
0807 15:07:30.584518 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:30.584550 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:30.585000 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:30.585190 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:30.585193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:30.585201 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:30.586105 (+ 904us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:30.586105 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:30.586134 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:30.586163 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:30.586314 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:30.586316 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:30.586319 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:30.586366 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:07:30.586412 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260807 15:07:30.587218 1521 heartbeater.cc:347] Connected to a master server at 127.18.129.62:44701
I20260807 15:07:30.587304 1521 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:30.587427 1521 heartbeater.cc:511] Master 127.18.129.62:44701 requested a full tablet report, sending...
I20260807 15:07:30.587888 1218 ts_manager.cc:194] Registered new tserver with Master: be9632e98b50469183e19f9703af9c32 (127.18.129.1:41265)
I20260807 15:07:30.588351 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 1303
I20260807 15:07:30.588433 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:07:30.588615 18948 external_mini_cluster.cc:1428] Setting key eeb7026cd4874121b854e1aecc59e15e
I20260807 15:07:30.589469 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:44701
--builtin_ntp_servers=127.18.129.20:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:07:30.672019 1524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:30.672223 1524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:30.672245 1524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:30.673346 1524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:30.673385 1524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:30.673408 1524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:30.674710 1524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:30.674927 1524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:30.675256 1524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:30.680593 1530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.680603 1532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.680605 1529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:30.680742 1524 server_base.cc:1034] running on GCE node
I20260807 15:07:30.681001 1524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:30.681241 1524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:30.682377 1524 hybrid_clock.cc:648] HybridClock initialized: now 1786115250682357 us; error 33 us; skew 500 ppm
I20260807 15:07:30.683640 1524 webserver.cc:460] Webserver started at http://127.18.129.2:39079/ using document root <none> and password file <none>
I20260807 15:07:30.683871 1524 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:30.683919 1524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:30.684051 1524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:30.685114 1524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "42c02638c68f4f869e41f25ead9c879e"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "d6d25592356ecd1be3c3b608c5a56bc8"
server_key_iv: "83523af119f2ff592ef5dc088667faf7"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.685446 1524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "42c02638c68f4f869e41f25ead9c879e"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "d6d25592356ecd1be3c3b608c5a56bc8"
server_key_iv: "83523af119f2ff592ef5dc088667faf7"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.686702 1524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:30.687561 1542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.687728 1524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:30.687815 1524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "42c02638c68f4f869e41f25ead9c879e"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "d6d25592356ecd1be3c3b608c5a56bc8"
server_key_iv: "83523af119f2ff592ef5dc088667faf7"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.687883 1524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:30.691190 1524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:30.691433 1524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:30.691550 1524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:30.691740 1524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:30.692020 1524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:30.692054 1524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.692080 1524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:30.692103 1524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.697506 1524 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:34793
I20260807 15:07:30.697543 1730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:34793 every 8 connection(s)
I20260807 15:07:30.697839 1524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:07:30.700753 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.698506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:39055 (local address 127.18.129.62:44701)
0807 15:07:30.698557 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:30.698558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:30.698792 (+ 234us) server_negotiation.cc:407] Connection header received
0807 15:07:30.699314 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:30.699315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:30.699322 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:30.699386 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:30.700489 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:30.700491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:30.700491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:30.700496 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:30.700514 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:30.700518 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:30.700586 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:30.700665 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 15:07:30.700680 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:07:30.700927 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.698485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:44701 (local address 127.18.129.2:39055)
0807 15:07:30.698680 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:30.698711 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:30.699214 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:30.699457 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:30.699461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:30.699470 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:30.700320 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:30.700321 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:30.700350 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:30.700378 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:30.700591 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:30.700597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:30.700600 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:30.700672 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:07:30.700733 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260807 15:07:30.701339 1731 heartbeater.cc:347] Connected to a master server at 127.18.129.62:44701
I20260807 15:07:30.701429 1731 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:30.701573 1731 heartbeater.cc:511] Master 127.18.129.62:44701 requested a full tablet report, sending...
I20260807 15:07:30.701946 1218 ts_manager.cc:194] Registered new tserver with Master: 42c02638c68f4f869e41f25ead9c879e (127.18.129.2:34793)
I20260807 15:07:30.704550 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 1524
I20260807 15:07:30.704643 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:07:30.704787 18948 external_mini_cluster.cc:1428] Setting key fcf87fb81f44e731c9e99c22ef8f41e2
I20260807 15:07:30.705616 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:44701
--builtin_ntp_servers=127.18.129.20:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:07:30.786707 1737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:30.786942 1737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:30.786963 1737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:30.788064 1737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:30.788103 1737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:30.788127 1737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:30.789397 1737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:44701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:30.789587 1737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:30.789827 1737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:30.794827 1742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.794899 1748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:30.794842 1743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:30.794919 1737 server_base.cc:1034] running on GCE node
I20260807 15:07:30.795261 1737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:30.795488 1737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:30.796627 1737 hybrid_clock.cc:648] HybridClock initialized: now 1786115250796615 us; error 30 us; skew 500 ppm
I20260807 15:07:30.797848 1737 webserver.cc:460] Webserver started at http://127.18.129.3:44239/ using document root <none> and password file <none>
I20260807 15:07:30.798082 1737 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:30.798135 1737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:30.798264 1737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:30.799381 1737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "70efca6d14954a20a3d4475a2924cf7d"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "26708e973753a50c14468345c288084b"
server_key_iv: "a66e56dd8e751b355ff270577da40977"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.799712 1737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "70efca6d14954a20a3d4475a2924cf7d"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "26708e973753a50c14468345c288084b"
server_key_iv: "a66e56dd8e751b355ff270577da40977"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.800902 1737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:07:30.801559 1755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.801712 1737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:30.801796 1737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "70efca6d14954a20a3d4475a2924cf7d"
format_stamp: "Formatted at 2026-08-07 15:07:30 on dist-test-slave-5bn2"
server_key: "26708e973753a50c14468345c288084b"
server_key_iv: "a66e56dd8e751b355ff270577da40977"
server_key_version: "encryptionkey@0"
I20260807 15:07:30.801856 1737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:30.805162 1737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:30.805401 1737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:30.805518 1737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:30.805701 1737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:30.805996 1737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:30.806035 1737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.806061 1737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:30.806094 1737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:30.811724 1737 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:40569
I20260807 15:07:30.811777 1934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:40569 every 8 connection(s)
I20260807 15:07:30.812076 1737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:07:30.814810 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.812739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:38951 (local address 127.18.129.62:44701)
0807 15:07:30.812798 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:30.812801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:30.813028 (+ 227us) server_negotiation.cc:407] Connection header received
0807 15:07:30.813483 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:30.813484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:30.813490 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:30.813539 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:30.814626 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:30.814627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:30.814628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:07:30.814632 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:30.814648 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:07:30.814652 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:30.814700 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:30.814744 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 15:07:30.814756 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:07:30.815012 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:30.812723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:44701 (local address 127.18.129.3:38951)
0807 15:07:30.812912 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:30.812945 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:30.813403 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:30.813598 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:30.813601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:30.813609 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:30.814477 (+ 868us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:07:30.814477 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:30.814506 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:07:30.814535 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:30.814699 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:30.814702 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:30.814704 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:07:30.814750 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:07:30.814794 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260807 15:07:30.815460 1935 heartbeater.cc:347] Connected to a master server at 127.18.129.62:44701
I20260807 15:07:30.815548 1935 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:30.815683 1935 heartbeater.cc:511] Master 127.18.129.62:44701 requested a full tablet report, sending...
I20260807 15:07:30.816020 1218 ts_manager.cc:194] Registered new tserver with Master: 70efca6d14954a20a3d4475a2924cf7d (127.18.129.3:40569)
I20260807 15:07:30.820736 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 1737
I20260807 15:07:30.820817 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:07:30.820974 18948 external_mini_cluster.cc:1428] Setting key 0c5aa4bd1d798f263e6ca96fe8a22261
I20260807 15:07:30.821825 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:07:30.823663 1300 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:07:30.823123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.18.129.62:44701)
0807 15:07:30.823164 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:30.823166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:30.823324 (+ 158us) server_negotiation.cc:407] Connection header received
0807 15:07:30.823399 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:30.823399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:30.823405 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:30.823448 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:30.823600 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57366: BlockingRecv error: recv got EOF from 127.0.0.1:57366 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:07:30.823697 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 1303
I20260807 15:07:30.847751 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 1524
I20260807 15:07:30.855085 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 1737
I20260807 15:07:30.865269 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 1165
2026-08-07T15:07:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (575 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:07:30 dist-test-slave-5bn2 krb5kdc[1952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:07:30 dist-test-slave-5bn2 krb5kdc[1952](info): set up 2 sockets
Aug 07 15:07:30 dist-test-slave-5bn2 krb5kdc[1952](info): commencing operation
krb5kdc: starting...
W20260807 15:07:32.025682 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:07:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:07:32.166016 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39703
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:39703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:32.247107 1968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:32.247349 1968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:32.247373 1968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:32.248548 1968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:07:32.248600 1968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:32.248615 1968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:32.248626 1968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:07:32.248646 1968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:07:32.250221 1968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:39703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:32.250460 1968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:32.250666 1968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:32.255710 1973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:32.255836 1968 server_base.cc:1034] running on GCE node
W20260807 15:07:32.255700 1981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:32.255673 1974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:32.256249 1968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:32.256579 1968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:32.257728 1968 hybrid_clock.cc:648] HybridClock initialized: now 1786115252257716 us; error 34 us; skew 500 ppm
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:32.260365 1968 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:07:32.260825 1968 webserver.cc:460] Webserver started at http://127.18.129.62:34227/ using document root <none> and password file <none>
I20260807 15:07:32.261061 1968 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:32.261113 1968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:32.261247 1968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:32.262342 1968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "fcd0ac63dd894ba1af0628180f905832"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "ffa095e3eca7ce0cb1d2d1acd13f8a9f"
server_key_iv: "c0be62b063e245063f82c9c413456a02"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.262702 1968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "fcd0ac63dd894ba1af0628180f905832"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "ffa095e3eca7ce0cb1d2d1acd13f8a9f"
server_key_iv: "c0be62b063e245063f82c9c413456a02"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.264137 1968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:07:32.264846 1992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.265009 1968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.265116 1968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "fcd0ac63dd894ba1af0628180f905832"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "ffa095e3eca7ce0cb1d2d1acd13f8a9f"
server_key_iv: "c0be62b063e245063f82c9c413456a02"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.265192 1968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:32.285523 1968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:32.287511 1968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:32.287672 1968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:32.291720 1968 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:39703
I20260807 15:07:32.291766 2070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:39703 every 8 connection(s)
I20260807 15:07:32.292189 1968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:07:32.293205 2079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:32.295619 2079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832: Bootstrap starting.
I20260807 15:07:32.296190 2079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:32.296476 2079 log.cc:824] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:32.297144 2079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832: No bootstrap required, opened a new log
I20260807 15:07:32.298810 2079 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd0ac63dd894ba1af0628180f905832" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39703 } }
I20260807 15:07:32.299001 2079 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:32.299041 2079 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcd0ac63dd894ba1af0628180f905832, State: Initialized, Role: FOLLOWER
I20260807 15:07:32.299190 2079 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd0ac63dd894ba1af0628180f905832" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39703 } }
I20260807 15:07:32.299254 2079 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:07:32.299276 2079 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:07:32.299324 2079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:32.300129 2079 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd0ac63dd894ba1af0628180f905832" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39703 } }
I20260807 15:07:32.300225 2079 leader_election.cc:302] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcd0ac63dd894ba1af0628180f905832; no voters:
I20260807 15:07:32.300395 2079 leader_election.cc:288] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:07:32.300432 2083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:32.300585 2083 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [term 1 LEADER]: Becoming Leader. State: Replica: fcd0ac63dd894ba1af0628180f905832, State: Running, Role: LEADER
I20260807 15:07:32.300649 2079 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:07:32.300680 2083 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd0ac63dd894ba1af0628180f905832" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39703 } }
I20260807 15:07:32.301007 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 1968
I20260807 15:07:32.301123 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:07:32.301326 18948 external_mini_cluster.cc:1428] Setting key d58abfc9c68de4269bf8fb86fb15a0b5
I20260807 15:07:32.302121 2084 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcd0ac63dd894ba1af0628180f905832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd0ac63dd894ba1af0628180f905832" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39703 } } }
I20260807 15:07:32.302244 2086 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcd0ac63dd894ba1af0628180f905832. Latest consensus state: current_term: 1 leader_uuid: "fcd0ac63dd894ba1af0628180f905832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd0ac63dd894ba1af0628180f905832" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39703 } } }
I20260807 15:07:32.302467 2086 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [sys.catalog]: This master's current role is: LEADER
I20260807 15:07:32.302533 2084 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832 [sys.catalog]: This master's current role is: LEADER
W20260807 15:07:32.302544 2110 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:07:32.302672 2110 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:07:32.302839 2114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:07:32.303062 2114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:07:32.305091 2114 catalog_manager.cc:1347] Generated new cluster ID: 33943eaf42314705baa5c791fcd6c5c1
I20260807 15:07:32.305137 2114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:32.313189 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.301801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37412 (local address 127.18.129.62:39703)
0807 15:07:32.302007 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:32.302018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:32.302667 (+ 649us) server_negotiation.cc:407] Connection header received
0807 15:07:32.302920 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:32.302928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:32.303023 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:32.303168 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:32.304001 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.304980 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.306051 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.306680 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.307135 (+ 455us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.309556 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:32.309560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:32.309560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:32.309574 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:32.311587 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.311947 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.311951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.311951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.312085 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.312259 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.312260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.312260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.312574 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:32.312724 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:32.312796 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 15:07:32.312949 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260807 15:07:32.314811 2114 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:07:32.315528 2114 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:07:32.323625 2114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832: Generated new TSK 0
I20260807 15:07:32.323822 2114 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:07:32.355155 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39703
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:32.439944 2120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:32.440150 2120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:32.440181 2120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:32.441304 2120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:32.441344 2120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:32.441380 2120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:07:32.442813 2120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:32.443068 2120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:32.443271 2120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:32.449045 2126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:32.449083 2120 server_base.cc:1034] running on GCE node
W20260807 15:07:32.449067 2131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:32.449062 2128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:32.449448 2120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:32.449723 2120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:32.450893 2120 hybrid_clock.cc:648] HybridClock initialized: now 1786115252450878 us; error 34 us; skew 500 ppm
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:32.453419 2120 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:07:32.453878 2120 webserver.cc:460] Webserver started at http://127.18.129.1:34693/ using document root <none> and password file <none>
I20260807 15:07:32.454066 2120 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:32.454104 2120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:32.454202 2120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:32.455246 2120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "25b2bd4c9ff64a7daa96408e7833de64"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "3ed735e1748929cf34a863403e5d78ba"
server_key_iv: "7fc5e275d0dbdb4d0926a2f7ce17c11e"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.455552 2120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "25b2bd4c9ff64a7daa96408e7833de64"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "3ed735e1748929cf34a863403e5d78ba"
server_key_iv: "7fc5e275d0dbdb4d0926a2f7ce17c11e"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.456772 2120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:32.457613 2141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.457801 2120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:07:32.457906 2120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "25b2bd4c9ff64a7daa96408e7833de64"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "3ed735e1748929cf34a863403e5d78ba"
server_key_iv: "7fc5e275d0dbdb4d0926a2f7ce17c11e"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.457989 2120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:32.481405 2120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:32.483618 2120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:32.483732 2120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:32.483924 2120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:32.484187 2120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:32.484218 2120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.484239 2120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:32.484251 2120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.489727 2120 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:46495
I20260807 15:07:32.489789 2313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:46495 every 8 connection(s)
I20260807 15:07:32.490087 2120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:07:32.491014 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 2120
I20260807 15:07:32.491130 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:07:32.491317 18948 external_mini_cluster.cc:1428] Setting key 14fd1fcb5ea303e51e82496a14775290
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:32.500295 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.490793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:50631 (local address 127.18.129.62:39703)
0807 15:07:32.490889 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:32.490892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:32.491049 (+ 157us) server_negotiation.cc:407] Connection header received
0807 15:07:32.491621 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:32.491623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:32.491638 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:32.491718 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:32.492713 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.493411 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.494454 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.494759 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.494841 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.497355 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:32.497356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:32.497356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:32.497420 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:32.498750 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.499241 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.499242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.499242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.499309 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.499658 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.499659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.499659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.499776 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:32.499824 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:32.500154 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 15:07:32.500225 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:07:32.500540 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.490735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39703 (local address 127.18.129.1:50631)
0807 15:07:32.490906 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:32.490948 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:32.491502 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:32.491800 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:32.491804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:32.492410 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:32.492607 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.492612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.493534 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.493535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.494359 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.494361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.494921 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.494922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.495044 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.495573 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:32.495582 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:32.497244 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:32.498833 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:32.498841 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:32.498843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:32.499160 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:32.499432 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:32.499433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:32.499433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:32.499584 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:32.499834 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:32.499840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:32.500007 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 15:07:32.500118 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:07:32.500293 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
I20260807 15:07:32.501060 2314 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39703
I20260807 15:07:32.501183 2314 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:32.501356 2314 heartbeater.cc:511] Master 127.18.129.62:39703 requested a full tablet report, sending...
I20260807 15:07:32.501874 2017 ts_manager.cc:194] Registered new tserver with Master: 25b2bd4c9ff64a7daa96408e7833de64 (127.18.129.1:46495)
I20260807 15:07:32.502537 2017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:50631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:07:32.519814 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39703
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:32.604941 2320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:32.605170 2320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:32.605204 2320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:32.606370 2320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:32.606421 2320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:32.606459 2320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:07:32.607971 2320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:32.608237 2320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:32.608465 2320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:32.613780 2328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:32.613826 2326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:32.613840 2320 server_base.cc:1034] running on GCE node
W20260807 15:07:32.613826 2325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:32.614171 2320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:32.614372 2320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:32.615562 2320 hybrid_clock.cc:648] HybridClock initialized: now 1786115252615541 us; error 33 us; skew 500 ppm
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:32.618031 2320 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:07:32.618471 2320 webserver.cc:460] Webserver started at http://127.18.129.2:45055/ using document root <none> and password file <none>
I20260807 15:07:32.618675 2320 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:32.618724 2320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:32.618877 2320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:32.619969 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "b4ebdd0b06264ae7a3720bea0abe9bca"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "628c351799536d554a4e14e7b8deda6c"
server_key_iv: "3bd3435fc6851011287ee458a802e8ca"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.620317 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "b4ebdd0b06264ae7a3720bea0abe9bca"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "628c351799536d554a4e14e7b8deda6c"
server_key_iv: "3bd3435fc6851011287ee458a802e8ca"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.621556 2320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:07:32.622217 2340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.622372 2320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:32.622462 2320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "b4ebdd0b06264ae7a3720bea0abe9bca"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "628c351799536d554a4e14e7b8deda6c"
server_key_iv: "3bd3435fc6851011287ee458a802e8ca"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.622529 2320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:32.664703 2320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:32.666960 2320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:32.667102 2320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:32.667385 2320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:32.667716 2320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:32.667753 2320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.667783 2320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:32.667807 2320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.673342 2320 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:45877
I20260807 15:07:32.673413 2500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:45877 every 8 connection(s)
I20260807 15:07:32.673698 2320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:07:32.675704 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 2320
I20260807 15:07:32.675789 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:07:32.675948 18948 external_mini_cluster.cc:1428] Setting key 48a61f3db379477f60643ecd92f4f046
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:32.682844 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.674394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:50313 (local address 127.18.129.62:39703)
0807 15:07:32.674441 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:32.674443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:32.674641 (+ 198us) server_negotiation.cc:407] Connection header received
0807 15:07:32.675219 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:32.675221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:32.675232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:32.675284 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:32.676324 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.676901 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.677885 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.678082 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.678148 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.680305 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:32.680306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:32.680307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:32.680315 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:32.681589 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.681948 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.681949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.681950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.682006 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.682219 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.682220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.682221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.682333 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:32.682391 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:32.682665 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 15:07:32.682753 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:07:32.683035 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.674354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39703 (local address 127.18.129.2:50313)
0807 15:07:32.674525 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:32.674562 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:32.675123 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:32.675355 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:32.675359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:32.675961 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:32.676238 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.676245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.676985 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.676986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.677803 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.677806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.678196 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.678197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.678358 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.678865 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:32.678876 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:32.680215 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:32.681666 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:32.681673 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:32.681675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:32.681880 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:32.682060 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:32.682060 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:32.682061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:32.682161 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:32.682389 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:32.682397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:32.682525 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 15:07:32.682632 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:07:32.682784 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260807 15:07:32.683476 2501 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39703
I20260807 15:07:32.683563 2501 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:32.683732 2501 heartbeater.cc:511] Master 127.18.129.62:39703 requested a full tablet report, sending...
I20260807 15:07:32.684136 2017 ts_manager.cc:194] Registered new tserver with Master: b4ebdd0b06264ae7a3720bea0abe9bca (127.18.129.2:45877)
I20260807 15:07:32.684494 2017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:50313
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:07:32.702586 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39703
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:07:32.783823 2506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:07:32.784029 2506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:07:32.784060 2506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:07:32.785174 2506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:07:32.785214 2506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:07:32.785248 2506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:07:32.786670 2506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:39703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:07:32.786935 2506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:07:32.787145 2506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:07:32.792189 2517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:07:32.792199 2521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:32.792271 2506 server_base.cc:1034] running on GCE node
W20260807 15:07:32.792210 2518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:07:32.792526 2506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:07:32.792727 2506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:07:32.793874 2506 hybrid_clock.cc:648] HybridClock initialized: now 1786115252793862 us; error 28 us; skew 500 ppm
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:07:32.796257 2506 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:07:32.796643 2506 webserver.cc:460] Webserver started at http://127.18.129.3:42021/ using document root <none> and password file <none>
I20260807 15:07:32.796835 2506 fs_manager.cc:359] Metadata directory not provided
I20260807 15:07:32.796875 2506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:07:32.796967 2506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:07:32.797979 2506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "3868ce479f7f403a9c40e8772fb4e759"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "eb3aed1fdb938d59332d69c5866e6068"
server_key_iv: "af1c1ed3685a4c45a497fba0b3d70ede"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.798327 2506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "3868ce479f7f403a9c40e8772fb4e759"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "eb3aed1fdb938d59332d69c5866e6068"
server_key_iv: "af1c1ed3685a4c45a497fba0b3d70ede"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.799647 2506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:07:32.800290 2532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.800426 2506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:07:32.800515 2506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "3868ce479f7f403a9c40e8772fb4e759"
format_stamp: "Formatted at 2026-08-07 15:07:32 on dist-test-slave-5bn2"
server_key: "eb3aed1fdb938d59332d69c5866e6068"
server_key_iv: "af1c1ed3685a4c45a497fba0b3d70ede"
server_key_version: "encryptionkey@0"
I20260807 15:07:32.800576 2506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:07:32.811702 2506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:07:32.813586 2506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:07:32.813714 2506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:07:32.813915 2506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:07:32.814208 2506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:07:32.814244 2506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.814272 2506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:07:32.814297 2506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:07:32.820044 2506 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:40635
I20260807 15:07:32.820118 2716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:40635 every 8 connection(s)
I20260807 15:07:32.820410 2506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:07:32 dist-test-slave-5bn2 krb5kdc[1952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115252, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:07:32.827562 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 2506
I20260807 15:07:32.827661 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:07:32.827788 18948 external_mini_cluster.cc:1428] Setting key c110c735f1b9a773190743efac444a42
I20260807 15:07:32.829702 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.821119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:45909 (local address 127.18.129.62:39703)
0807 15:07:32.821179 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:32.821181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:32.821368 (+ 187us) server_negotiation.cc:407] Connection header received
0807 15:07:32.821952 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:32.821953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:32.821966 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:32.822039 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:32.823021 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.823521 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.824596 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.824789 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.824843 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.826948 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:32.826949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:32.826950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:32.826956 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:32.828380 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.828932 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.828933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.828933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.828991 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.829216 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.829216 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.829217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.829315 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:32.829360 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:32.829577 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 15:07:32.829639 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:07:32.829869 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.821064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:39703 (local address 127.18.129.3:45909)
0807 15:07:32.821250 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:32.821287 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:32.821838 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:32.822111 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:32.822116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:32.822694 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:07:32.822922 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.822927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.823605 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.823606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.824512 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.824515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.824859 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.824859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.824990 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.825488 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:07:32.825497 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:07:32.826841 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:07:32.828582 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:32.828590 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:32.828592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:32.828864 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:32.829044 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:07:32.829045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:07:32.829045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:07:32.829169 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:07:32.829359 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:07:32.829362 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:07:32.829476 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:07:32.829552 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:07:32.829676 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260807 15:07:32.830267 2718 heartbeater.cc:347] Connected to a master server at 127.18.129.62:39703
I20260807 15:07:32.830353 2718 heartbeater.cc:464] Registering TS with master...
I20260807 15:07:32.830531 2718 heartbeater.cc:511] Master 127.18.129.62:39703 requested a full tablet report, sending...
I20260807 15:07:32.830946 2017 ts_manager.cc:194] Registered new tserver with Master: 3868ce479f7f403a9c40e8772fb4e759 (127.18.129.3:40635)
I20260807 15:07:32.831317 2017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:45909
I20260807 15:07:32.839526 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:07:32.846434 2103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.840763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.18.129.62:39703)
0807 15:07:32.840817 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:32.840819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:32.840904 (+ 85us) server_negotiation.cc:407] Connection header received
0807 15:07:32.840968 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:32.840969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:32.840979 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:32.841025 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:07:32.841564 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.842032 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.842747 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.842979 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.843035 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.844133 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:07:32.844135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:07:32.844135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:07:32.844142 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:07:32.845395 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.845704 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.845705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.845705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.845761 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:07:32.845918 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:07:32.845919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:07:32.845919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:07:32.846071 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:07:32.846162 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:32.846267 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:07:32.846341 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:07:32.847882 2017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:07:32.849473 2017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:07:32.855971 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.852109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:40635 (local address 127.0.0.1:35536)
0807 15:07:32.852353 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:32.852360 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:32.852392 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:32.852713 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:32.852714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:32.852716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:32.852763 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.852764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.853651 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.853652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.854885 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.854888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.855764 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.855765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.855820 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.855823 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:32.855868 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:07:32.855901 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":64,"threads_started":1}
I20260807 15:07:32.856004 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.852212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.18.129.3:40635)
0807 15:07:32.852494 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:32.852499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:32.852508 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:07:32.852546 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:32.852547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:32.852572 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:32.852643 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:32.852827 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.853552 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.854958 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.855692 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.855768 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.855825 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:32.855884 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:07:32.855924 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":76,"threads_started":1}
I20260807 15:07:32.856065 2735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.851867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52772 (local address 127.18.129.1:46495)
0807 15:07:32.852508 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:32.852512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:32.852520 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:07:32.852560 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:32.852561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:32.852587 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:32.852660 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:32.852856 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.853907 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.855013 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.855549 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.855621 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.855685 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:32.855944 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 15:07:32.855982 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":101,"threads_started":1}
I20260807 15:07:32.856177 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.851743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:46495 (local address 127.0.0.1:52772)
0807 15:07:32.851868 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:32.851914 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:32.851983 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:32.852745 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:32.852745 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:32.852749 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:32.852785 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.852787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.854004 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.854005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.854927 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.854931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.855653 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.855654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.855734 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.855749 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:07:32.855815 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:07:32.855869 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260807 15:07:32.856917 2737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.851934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:45877 (local address 127.0.0.1:48404)
0807 15:07:32.852229 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:32.852240 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:32.852287 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:32.852557 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:32.852559 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:32.852561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:32.852632 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.852634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.853969 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.853970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.855061 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:32.855063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.855984 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.855985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:32.856741 (+ 756us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.856747 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:07:32.856808 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:07:32.856849 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":86,"threads_started":1}
I20260807 15:07:32.856820 2605 tablet_service.cc:1467] Processing CreateTablet for tablet 6e9f56e36ad7417e878446ec278b450a (DEFAULT_TABLE table=test-table [id=111f1b05a45a408182f0a0007bb5a609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:32.856900 2211 tablet_service.cc:1467] Processing CreateTablet for tablet 6e9f56e36ad7417e878446ec278b450a (DEFAULT_TABLE table=test-table [id=111f1b05a45a408182f0a0007bb5a609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:32.857043 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:32.852060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.18.129.2:45877)
0807 15:07:32.852285 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:32.852290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:32.852300 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:07:32.852347 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:32.852349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:32.852388 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:32.852460 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:32.852711 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.853860 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.855137 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:32.855838 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:32.855898 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:32.855948 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:32.856841 (+ 893us) server_negotiation.cc:299] Negotiation successful
0807 15:07:32.856888 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
I20260807 15:07:32.857220 2605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e9f56e36ad7417e878446ec278b450a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:32.857283 2211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e9f56e36ad7417e878446ec278b450a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:32.859548 2408 tablet_service.cc:1467] Processing CreateTablet for tablet 6e9f56e36ad7417e878446ec278b450a (DEFAULT_TABLE table=test-table [id=111f1b05a45a408182f0a0007bb5a609]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:07:32.859953 2408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e9f56e36ad7417e878446ec278b450a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:07:32.859977 2741 tablet_bootstrap.cc:492] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Bootstrap starting.
I20260807 15:07:32.860670 2741 tablet_bootstrap.cc:654] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:32.861119 2741 log.cc:824] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:32.861924 2741 tablet_bootstrap.cc:492] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: No bootstrap required, opened a new log
I20260807 15:07:32.861981 2741 ts_tablet_manager.cc:1397] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:07:32.862793 2747 tablet_bootstrap.cc:492] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Bootstrap starting.
I20260807 15:07:32.863121 2742 tablet_bootstrap.cc:492] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64: Bootstrap starting.
I20260807 15:07:32.863536 2741 raft_consensus.cc:348] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:32.863678 2741 raft_consensus.cc:374] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:32.863711 2741 raft_consensus.cc:729] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3868ce479f7f403a9c40e8772fb4e759, State: Initialized, Role: FOLLOWER
I20260807 15:07:32.863798 2742 tablet_bootstrap.cc:654] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:32.863798 2747 tablet_bootstrap.cc:654] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Neither blocks nor log segments found. Creating new log.
I20260807 15:07:32.863845 2741 consensus_queue.cc:260] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:32.864125 2741 ts_tablet_manager.cc:1428] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:07:32.864125 2718 heartbeater.cc:503] Master 127.18.129.62:39703 was elected leader, sending a full tablet report...
I20260807 15:07:32.864360 2742 log.cc:824] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:32.864390 2747 log.cc:824] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Log is configured to *not* fsync() on all Append() calls
I20260807 15:07:32.865087 2742 tablet_bootstrap.cc:492] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64: No bootstrap required, opened a new log
I20260807 15:07:32.865161 2742 ts_tablet_manager.cc:1397] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:07:32.866518 2742 raft_consensus.cc:348] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:32.866685 2742 raft_consensus.cc:374] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:32.866734 2742 raft_consensus.cc:729] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25b2bd4c9ff64a7daa96408e7833de64, State: Initialized, Role: FOLLOWER
I20260807 15:07:32.866879 2742 consensus_queue.cc:260] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:32.867170 2742 ts_tablet_manager.cc:1428] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:07:32.867198 2747 tablet_bootstrap.cc:492] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: No bootstrap required, opened a new log
I20260807 15:07:32.867182 2314 heartbeater.cc:503] Master 127.18.129.62:39703 was elected leader, sending a full tablet report...
I20260807 15:07:32.867262 2747 ts_tablet_manager.cc:1397] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 15:07:32.868887 2747 raft_consensus.cc:348] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:32.869009 2747 raft_consensus.cc:374] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:07:32.869029 2747 raft_consensus.cc:729] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4ebdd0b06264ae7a3720bea0abe9bca, State: Initialized, Role: FOLLOWER
I20260807 15:07:32.869110 2747 consensus_queue.cc:260] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:32.869349 2747 ts_tablet_manager.cc:1428] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:07:32.869413 2501 heartbeater.cc:503] Master 127.18.129.62:39703 was elected leader, sending a full tablet report...
W20260807 15:07:32.924269 2502 tablet.cc:2369] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:07:32.990824 2315 tablet.cc:2369] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:07:33.071036 2720 tablet.cc:2369] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:07:33.107016 2754 raft_consensus.cc:482] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:07:33.107175 2754 raft_consensus.cc:504] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:33.107594 2754 leader_election.cc:288] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25b2bd4c9ff64a7daa96408e7833de64 (127.18.129.1:46495), 3868ce479f7f403a9c40e8772fb4e759 (127.18.129.3:40635)
I20260807 15:07:33.110452 2735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:33.107887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:40777 (local address 127.18.129.1:46495)
0807 15:07:33.107942 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:33.107943 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:33.107956 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:07:33.108036 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:33.108037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:33.108049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:33.108120 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:33.108290 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.108784 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.109760 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.110133 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.110194 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:33.110204 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:33.110341 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:07:33.110378 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:07:33.110479 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:33.107806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:46495 (local address 127.18.129.2:40777)
0807 15:07:33.107885 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:33.107898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:33.107949 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:33.108131 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:33.108132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:33.108134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:33.108206 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:33.108208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.108874 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.108875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:33.109670 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:33.109672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.110207 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.110209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:33.110278 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:33.110281 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:33.110326 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:07:33.110374 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":20}
I20260807 15:07:33.110502 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:33.108014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:38861 (local address 127.18.129.3:40635)
0807 15:07:33.108077 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:33.108079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:33.108226 (+ 147us) server_negotiation.cc:407] Connection header received
0807 15:07:33.108260 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:33.108261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:33.108271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:33.108317 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:07:33.108462 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.108922 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.109800 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.110186 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.110250 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:33.110259 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:33.110401 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:07:33.110437 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:07:33.110721 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:33.107947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:40635 (local address 127.18.129.2:38861)
0807 15:07:33.108149 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:33.108160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:07:33.108217 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:07:33.108332 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:07:33.108333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:07:33.108334 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:07:33.108405 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:33.108407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.108996 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.108997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:33.109719 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:07:33.109721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.110265 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.110266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:07:33.110339 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:33.110342 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:07:33.110387 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:07:33.110423 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":122,"threads_started":1}
I20260807 15:07:33.110832 2242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e9f56e36ad7417e878446ec278b450a" candidate_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25b2bd4c9ff64a7daa96408e7833de64" is_pre_election: true
I20260807 15:07:33.111021 2242 raft_consensus.cc:2454] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4ebdd0b06264ae7a3720bea0abe9bca in term 0.
I20260807 15:07:33.111141 2639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e9f56e36ad7417e878446ec278b450a" candidate_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3868ce479f7f403a9c40e8772fb4e759" is_pre_election: true
I20260807 15:07:33.111253 2342 leader_election.cc:302] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25b2bd4c9ff64a7daa96408e7833de64, b4ebdd0b06264ae7a3720bea0abe9bca; no voters:
I20260807 15:07:33.111325 2639 raft_consensus.cc:2454] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4ebdd0b06264ae7a3720bea0abe9bca in term 0.
I20260807 15:07:33.111392 2754 raft_consensus.cc:2781] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:07:33.111469 2754 raft_consensus.cc:482] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:07:33.111501 2754 raft_consensus.cc:3037] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:33.112242 2754 raft_consensus.cc:504] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:33.112370 2754 leader_election.cc:288] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [CANDIDATE]: Term 1 election: Requested vote from peers 25b2bd4c9ff64a7daa96408e7833de64 (127.18.129.1:46495), 3868ce479f7f403a9c40e8772fb4e759 (127.18.129.3:40635)
I20260807 15:07:33.112533 2242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e9f56e36ad7417e878446ec278b450a" candidate_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25b2bd4c9ff64a7daa96408e7833de64"
I20260807 15:07:33.112567 2639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e9f56e36ad7417e878446ec278b450a" candidate_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3868ce479f7f403a9c40e8772fb4e759"
I20260807 15:07:33.112625 2242 raft_consensus.cc:3037] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:33.112632 2639 raft_consensus.cc:3037] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:07:33.113317 2242 raft_consensus.cc:2454] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4ebdd0b06264ae7a3720bea0abe9bca in term 1.
I20260807 15:07:33.113344 2639 raft_consensus.cc:2454] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4ebdd0b06264ae7a3720bea0abe9bca in term 1.
I20260807 15:07:33.113480 2342 leader_election.cc:302] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25b2bd4c9ff64a7daa96408e7833de64, b4ebdd0b06264ae7a3720bea0abe9bca; no voters:
I20260807 15:07:33.113570 2754 raft_consensus.cc:2781] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:07:33.113729 2754 raft_consensus.cc:686] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 1 LEADER]: Becoming Leader. State: Replica: b4ebdd0b06264ae7a3720bea0abe9bca, State: Running, Role: LEADER
I20260807 15:07:33.113821 2754 consensus_queue.cc:237] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } }
I20260807 15:07:33.114656 2017 catalog_manager.cc:5495] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca reported cstate change: term changed from 0 to 1, leader changed from <none> to b4ebdd0b06264ae7a3720bea0abe9bca (127.18.129.2). New cstate: current_term: 1 leader_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4ebdd0b06264ae7a3720bea0abe9bca" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 } health_report { overall_health: UNKNOWN } } }
I20260807 15:07:33.140545 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:33.138108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.18.129.2:45877)
0807 15:07:33.138186 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:07:33.138187 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:07:33.138196 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:07:33.138240 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:07:33.138241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:07:33.138264 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:07:33.138343 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:07:33.138488 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.139092 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.139830 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:07:33.140055 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:07:33.140112 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:07:33.140258 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:07:33.140363 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:07:33.140422 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:07:33.140436 (+ 14us) server_negotiation.cc:299] Negotiation successful
0807 15:07:33.140455 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:07:33.142359 2639 raft_consensus.cc:1261] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 1 FOLLOWER]: Refusing update from remote peer b4ebdd0b06264ae7a3720bea0abe9bca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:33.142382 2242 raft_consensus.cc:1261] T 6e9f56e36ad7417e878446ec278b450a P 25b2bd4c9ff64a7daa96408e7833de64 [term 1 FOLLOWER]: Refusing update from remote peer b4ebdd0b06264ae7a3720bea0abe9bca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:07:33.142678 2754 consensus_queue.cc:1035] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3868ce479f7f403a9c40e8772fb4e759" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:33.142827 2760 consensus_queue.cc:1035] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "25b2bd4c9ff64a7daa96408e7833de64" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:07:33.145738 2765 mvcc.cc:205] Tried to move back new op lower bound from 7315928076868435968 to 7315928076754657280. Current Snapshot: MvccSnapshot[applied={T|T < 7315928076868435968 or (T in {7315928076868435968})}]
I20260807 15:07:33.146028 2766 mvcc.cc:205] Tried to move back new op lower bound from 7315928076868435968 to 7315928076754657280. Current Snapshot: MvccSnapshot[applied={T|T < 7315928076868435968}]
I20260807 15:07:33.148226 2015 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:33.148329 2015 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:07:33.149242 2015 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcd0ac63dd894ba1af0628180f905832: Sending DeleteTablet for 3 replicas of tablet 6e9f56e36ad7417e878446ec278b450a
I20260807 15:07:33.149621 2408 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e9f56e36ad7417e878446ec278b450a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:33 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:48404
I20260807 15:07:33.149658 2211 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e9f56e36ad7417e878446ec278b450a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:33 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:52772
I20260807 15:07:33.149899 2605 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e9f56e36ad7417e878446ec278b450a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:07:33 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:35536
I20260807 15:07:33.149951 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 2120
I20260807 15:07:33.162212 2783 tablet_replica.cc:330] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: stopping tablet replica
I20260807 15:07:33.162379 2783 raft_consensus.cc:2229] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:07:33.162521 2783 raft_consensus.cc:2258] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:33.162902 2782 tablet_replica.cc:330] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: stopping tablet replica
I20260807 15:07:33.163018 2782 raft_consensus.cc:2229] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:07:33.163133 2782 raft_consensus.cc:2258] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:07:33.164798 2782 ts_tablet_manager.cc:1905] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:33.165481 2783 ts_tablet_manager.cc:1905] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:07:33.166649 2782 ts_tablet_manager.cc:1918] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:33.166733 2782 log.cc:1196] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/6e9f56e36ad7417e878446ec278b450a
I20260807 15:07:33.167060 2782 ts_tablet_manager.cc:1939] T 6e9f56e36ad7417e878446ec278b450a P b4ebdd0b06264ae7a3720bea0abe9bca: Deleting consensus metadata
I20260807 15:07:33.167075 2783 ts_tablet_manager.cc:1918] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:07:33.167138 2783 log.cc:1196] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/6e9f56e36ad7417e878446ec278b450a
I20260807 15:07:33.167425 2783 ts_tablet_manager.cc:1939] T 6e9f56e36ad7417e878446ec278b450a P 3868ce479f7f403a9c40e8772fb4e759: Deleting consensus metadata
I20260807 15:07:33.167626 1995 catalog_manager.cc:4841] TS b4ebdd0b06264ae7a3720bea0abe9bca (127.18.129.2:45877): tablet 6e9f56e36ad7417e878446ec278b450a (table test-table [id=111f1b05a45a408182f0a0007bb5a609]) successfully deleted
I20260807 15:07:33.167819 1995 catalog_manager.cc:4841] TS 3868ce479f7f403a9c40e8772fb4e759 (127.18.129.3:40635): tablet 6e9f56e36ad7417e878446ec278b450a (table test-table [id=111f1b05a45a408182f0a0007bb5a609]) successfully deleted
W20260807 15:07:33.176177 1995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:46495 (error 108)
I20260807 15:07:33.176795 2737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:07:33.176650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:46495 (local address 127.0.0.1:52786)
0807 15:07:33.176710 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 15:07:33.176756 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:07:33.176800 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 2320
W20260807 15:07:33.184466 1995 catalog_manager.cc:4584] TS 25b2bd4c9ff64a7daa96408e7833de64 (127.18.129.1:46495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e9f56e36ad7417e878446ec278b450a: Network error: Client connection negotiation failed: client connection to 127.18.129.1:46495: connect: Connection refused (error 111)
I20260807 15:07:33.188774 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 2506
I20260807 15:07:33.199112 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 1968
2026-08-07T15:07:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2323 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:07:33 dist-test-slave-5bn2 krb5kdc[2787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:07:33 dist-test-slave-5bn2 krb5kdc[2787](info): set up 2 sockets
Aug 07 15:07:33 dist-test-slave-5bn2 krb5kdc[2787](info): commencing operation
krb5kdc: starting...
W20260807 15:07:34.590052 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.367s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:07:34 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:07:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.129.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.129.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/rangeradmin_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/rangeradmin_127.18.129.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.129.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.129.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/rangerlookup_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/rangerlookup_127.18.129.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.129.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.129.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/HTTP_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/HTTP_127.18.129.20@KRBTEST.COM.keytab.
I20260807 15:07:34.851303 18948 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDKyZDM/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/postgres -l logfile start
2026-08-07 15:07:37.985 UTC [2833] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:07:37.985 UTC [2833] LOG: listening on IPv4 address "127.18.129.20", port 39973
2026-08-07 15:07:37.988 UTC [2833] LOG: listening on Unix socket "/tmp/.s.PGSQL.39973"
2026-08-07 15:07:38.009 UTC [2836] LOG: database system was shut down at 2026-08-07 15:07:36 UTC
2026-08-07 15:07:38.013 UTC [2833] LOG: database system is ready to accept connections
I20260807 15:07:39.067054 18948 mini_postgres.cc:96] Postgres bound to 39973
2026-08-07 15:07:39.075 UTC [2851] FATAL: database "slave" does not exist
127.18.129.20:39973 - accepting connections
I20260807 15:07:39.076442 18948 mini_ranger.cc:161] Starting Ranger...
I20260807 15:07:39.126794 18948 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:07:39.580735 18948 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:07:39.580829 18948 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-admin
2026-08-07 15:07:39,944 [I] DB FLAVOR :POSTGRES
2026-08-07 15:07:39,944 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:07:39,945 [I] Checking connection..
2026-08-07 15:07:39,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:07:40,218 [I] Checking connection passed.
2026-08-07 15:07:40,218 [I] --------- Verifying version history table ---------
2026-08-07 15:07:40,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:07:40,482 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:07:40,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:07:40,746 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:07:40,747 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:07:40,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:07:41.002 UTC [2956] WARNING: there is no transaction in progress
2026-08-07 15:07:41,009 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:07:41,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:07:41,274 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:07:41,275 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:07:41,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:07:41,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:07:41,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5bn2.c.gcp-upstream.internal','N') ;"
2026-08-07 15:07:42,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:07:42,326 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:07:42,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:07:42,577 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:07:42,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:07:42,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:07:43,094 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:07:43,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:07:44.056 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.073 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.088 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.103 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.120 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.141 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.205 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.211 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.225 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.234 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.244 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.258 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.268 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.274 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.280 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44.287 UTC [3121] WARNING: there is no transaction in progress
2026-08-07 15:07:44,901 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:07:44,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:07:45,134 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:07:45,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:07:45,387 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:07:45,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:07:45,655 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:07:45,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:07:45,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:07:46,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5bn2.c.gcp-upstream.internal';"
2026-08-07 15:07:46,425 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:07:46,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:07:46,682 [I] DB_PATCHES have already been applied
I20260807 15:07:46.688423 18948 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDKyZDM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:07:46.688493 18948 mini_ranger.cc:193] Using host: 127.18.129.20
I20260807 15:07:46.692129 18948 mini_ranger.cc:227] Ranger admin URL: http://127.18.129.20:40907
Aug 07, 2026 3:07:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:07:49.184994 18948 mini_ranger.cc:160] Time spent starting Ranger: real 10.109s user 0.004s sys 0.009s
Aug 07, 2026 3:08:09 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:08:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 3:08:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:08:09 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:08:10 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:08:11.116117 18948 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.129.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.129.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/rangerkms_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/rangerkms_127.18.129.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.129.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.129.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.129.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/HTTP_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.129.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/HTTP_127.18.129.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:08:11 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115291, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:08:11.204201 18948 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:08:11.215066 18948 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:08:12.041195 18948 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:08:12.041285 18948 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms
I20260807 15:08:12.041302 18948 mini_ranger_kms.cc:227] Using postgres at 127.18.129.20:39973
2026-08-07 15:08:12,559 [I] DB FLAVOR :POSTGRES
2026-08-07 15:08:12,559 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:08:12,559 [I] Checking connection
2026-08-07 15:08:12,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:08:12,879 [I] connection success
2026-08-07 15:08:12,880 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:08:12,880 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:08:12,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:08:13,128 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:08:13,128 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:08:13,128 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:08:13,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:39973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:08:13,383 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:08:13.850641 18948 mini_ranger_kms.cc:310] Created kms service
I20260807 15:08:14.070292 18948 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:08:14.111356 18948 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:08:14 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115291, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:08:14.521358 18948 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:08:14.521487 18948 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:08:14.521510 18948 mini_ranger_kms.cc:240] Using host: 127.18.129.20
I20260807 15:08:14.526326 18948 mini_ranger_kms.cc:276] Ranger KMS PID: 3398
I20260807 15:08:14.526435 18948 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.129.20:41555
Aug 07, 2026 3:08:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:08:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:08:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:08:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:08:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:08:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:08:16 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115296, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.129.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:08:17.003336 18948 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.799s user 0.000s sys 0.490s
I20260807 15:08:17.003470 18948 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:08:17.003568 18948 mini_ranger_kms.cc:401] 127.18.129.20:41555/kms/v1/keys
Aug 07 15:08:17 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115296, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.129.20@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
Aug 07 15:08:17 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115296, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.129.20@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
Aug 07 15:08:20 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:08:20.429942 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46291
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46291
--ranger_config_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:41555/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:08:20.558755 3467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:08:20.559027 3467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:08:20.559051 3467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:08:20.560357 3467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:08:20.560405 3467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:08:20.560418 3467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:08:20.560441 3467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:08:20.560463 3467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:08:20.562695 3467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.129.20:41555/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46291
--ranger_config_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:08:20.563002 3467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:08:20.563275 3467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:08:20.568747 3476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:08:20.568763 3477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:08:20.568792 3467 server_base.cc:1034] running on GCE node
W20260807 15:08:20.568763 3479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:08:20.569183 3467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:08:20.569475 3467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:08:20.570632 3467 hybrid_clock.cc:648] HybridClock initialized: now 1786115300570620 us; error 39 us; skew 500 ppm
Aug 07 15:08:20 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:08:20.573165 3467 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:08:20.573577 3467 webserver.cc:460] Webserver started at http://127.18.129.62:44387/ using document root <none> and password file <none>
I20260807 15:08:20.573786 3467 fs_manager.cc:359] Metadata directory not provided
I20260807 15:08:20.573827 3467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:08:20.573956 3467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:08:20 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:08:20.623843 3467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5"
format_stamp: "Formatted at 2026-08-07 15:08:20 on dist-test-slave-5bn2"
server_key: "1a29aba3643b2d1365918f11efed0fb8"
server_key_iv: "22f4b9d8d61ef39e6011e1447f6fc00c"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:20.624336 3467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5"
format_stamp: "Formatted at 2026-08-07 15:08:20 on dist-test-slave-5bn2"
server_key: "1a29aba3643b2d1365918f11efed0fb8"
server_key_iv: "22f4b9d8d61ef39e6011e1447f6fc00c"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:20.626674 3467 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:08:20.643604 3491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:20.643872 3467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:08:20.644019 3467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5"
format_stamp: "Formatted at 2026-08-07 15:08:20 on dist-test-slave-5bn2"
server_key: "1a29aba3643b2d1365918f11efed0fb8"
server_key_iv: "22f4b9d8d61ef39e6011e1447f6fc00c"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:20.644124 3467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:08:20.666702 3467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:08:20.668998 3467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:08:20.669252 3467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:08:20.684782 3467 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:46291
I20260807 15:08:20.684886 3573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:46291 every 8 connection(s)
I20260807 15:08:20.685434 3467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:08:20.686609 3575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:08:20.687408 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 3467
I20260807 15:08:20.687513 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:08:20.687733 18948 external_mini_cluster.cc:1428] Setting key 300381894e1107394fbba53bc5c72592
I20260807 15:08:20.691020 3575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5: Bootstrap starting.
I20260807 15:08:20.692299 3575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5: Neither blocks nor log segments found. Creating new log.
I20260807 15:08:20.693022 3575 log.cc:824] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:08:20.694017 3575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5: No bootstrap required, opened a new log
Aug 07 15:08:20 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:08:20.698261 3575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46291 } }
I20260807 15:08:20.698426 3575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:08:20.698499 3575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93aa4a1cf13d46aeb954c4c7fb5999d5, State: Initialized, Role: FOLLOWER
I20260807 15:08:20.698730 3575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46291 } }
I20260807 15:08:20.698818 3575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:08:20.698869 3575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:08:20.698936 3575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:08:20.699620 3575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46291 } }
I20260807 15:08:20.699719 3575 leader_election.cc:302] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93aa4a1cf13d46aeb954c4c7fb5999d5; no voters:
I20260807 15:08:20.699891 3575 leader_election.cc:288] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:08:20.700059 3583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:08:20.700417 3575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:08:20.700417 3583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [term 1 LEADER]: Becoming Leader. State: Replica: 93aa4a1cf13d46aeb954c4c7fb5999d5, State: Running, Role: LEADER
I20260807 15:08:20.701010 3583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46291 } }
I20260807 15:08:20.702350 3587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93aa4a1cf13d46aeb954c4c7fb5999d5. Latest consensus state: current_term: 1 leader_uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46291 } } }
I20260807 15:08:20.702440 3587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:08:20.702740 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:20.688680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36958 (local address 127.18.129.62:46291)
0807 15:08:20.689055 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:20.689065 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:20.689125 (+ 60us) server_negotiation.cc:407] Connection header received
0807 15:08:20.689977 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:20.689987 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:20.690054 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:20.690272 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:08:20.691193 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:20.692737 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:20.693703 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:20.694399 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:20.694508 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:20.697907 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:08:20.697914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:08:20.697915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:08:20.697929 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:08:20.700488 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:20.700886 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:20.700896 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:20.700897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:20.701000 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:20.701225 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:20.701227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:20.701227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:20.701560 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:08:20.701704 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:20.702235 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 15:08:20.702370 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":187,"threads_started":1}
I20260807 15:08:20.703135 3603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:08:20.703347 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aa4a1cf13d46aeb954c4c7fb5999d5" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46291 } } }
I20260807 15:08:20.703421 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:08:20.704950 3603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:08:20.705425 3575 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:08:20.713146 3603 catalog_manager.cc:1347] Generated new cluster ID: 5318eed6eaac45c0bb126c3ea1948b1c
I20260807 15:08:20.713460 3603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:08:20.722220 3603 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:08:20.723047 3603 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:08:20.728457 3603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5: Generated new TSK 0
I20260807 15:08:20.728718 3603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:08:21.626037 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:41555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46291
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:08:21.776921 3646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:08:21.777153 3646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:08:21.777189 3646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:08:21.778432 3646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:08:21.778473 3646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:08:21.778510 3646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:08:21.780102 3646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.129.20:41555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:08:21.780351 3646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:08:21.780576 3646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:08:21.786043 3651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:08:21.786116 3652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:08:21.786468 3654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:08:21.786723 3646 server_base.cc:1034] running on GCE node
I20260807 15:08:21.786942 3646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:08:21.787171 3646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:08:21.788321 3646 hybrid_clock.cc:648] HybridClock initialized: now 1786115301788307 us; error 32 us; skew 500 ppm
Aug 07 15:08:21 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:08:21.791674 3646 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:08:21.792151 3646 webserver.cc:460] Webserver started at http://127.18.129.1:44859/ using document root <none> and password file <none>
I20260807 15:08:21.792383 3646 fs_manager.cc:359] Metadata directory not provided
I20260807 15:08:21.792431 3646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:08:21.792557 3646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:08:21 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:08:21.807965 3646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff"
format_stamp: "Formatted at 2026-08-07 15:08:21 on dist-test-slave-5bn2"
server_key: "6bac93992d9ec87795b7fb5e411303aa"
server_key_iv: "5503a145f29ffd36a6e4dc3e7c0f06c9"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:21.808544 3646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff"
format_stamp: "Formatted at 2026-08-07 15:08:21 on dist-test-slave-5bn2"
server_key: "6bac93992d9ec87795b7fb5e411303aa"
server_key_iv: "5503a145f29ffd36a6e4dc3e7c0f06c9"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:21.810909 3646 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260807 15:08:21.826502 3664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:21.826905 3646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:08:21.827091 3646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff"
format_stamp: "Formatted at 2026-08-07 15:08:21 on dist-test-slave-5bn2"
server_key: "6bac93992d9ec87795b7fb5e411303aa"
server_key_iv: "5503a145f29ffd36a6e4dc3e7c0f06c9"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:21.827246 3646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:08:21.858060 3646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:08:21.860829 3646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:08:21.861124 3646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:08:21.861545 3646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:08:21.862059 3646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:08:21.862112 3646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:21.862142 3646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:08:21.862159 3646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:21.881491 3646 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:45293
I20260807 15:08:21.882222 3646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:08:21.883220 3845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:45293 every 8 connection(s)
I20260807 15:08:21.887703 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 3646
I20260807 15:08:21.887826 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:08:21.888051 18948 external_mini_cluster.cc:1428] Setting key 4186b9b307b4e25dbf9dd1746b392980
Aug 07 15:08:21 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115301, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:08:21.897113 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:21.884016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46291 (local address 127.18.129.1:60433)
0807 15:08:21.884337 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:21.884393 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:08:21.885614 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:08:21.886079 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:08:21.886084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:08:21.886613 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:08:21.887050 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:21.887058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:21.888113 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:21.888114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:21.889561 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:21.889564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:21.890062 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:21.890063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:21.890256 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:21.891008 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:08:21.891019 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:08:21.893262 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:08:21.895293 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:08:21.895301 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:08:21.895303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:08:21.895599 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:08:21.895828 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:08:21.895829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:08:21.895830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:08:21.895972 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:08:21.896312 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:08:21.896319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:08:21.896491 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 15:08:21.896612 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 15:08:21.896781 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":190,"threads_started":1}
I20260807 15:08:21.897706 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:21.884058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:60433 (local address 127.18.129.62:46291)
0807 15:08:21.884436 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:21.884438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:21.884500 (+ 62us) server_negotiation.cc:407] Connection header received
0807 15:08:21.885816 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:21.885818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:21.885835 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:21.885920 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:08:21.887173 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:21.887970 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:21.889654 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:21.889916 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:21.889976 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:21.893391 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:08:21.893393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:08:21.893393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:08:21.893402 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:08:21.895199 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:21.895685 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:21.895687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:21.895687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:21.895755 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:21.896066 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:21.896067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:21.896067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:21.896217 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:08:21.896282 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:21.897501 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0807 15:08:21.897596 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":69,"threads_started":1}
I20260807 15:08:21.898877 3846 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46291
I20260807 15:08:21.899003 3846 heartbeater.cc:464] Registering TS with master...
I20260807 15:08:21.899214 3846 heartbeater.cc:511] Master 127.18.129.62:46291 requested a full tablet report, sending...
I20260807 15:08:21.899910 3516 ts_manager.cc:194] Registered new tserver with Master: 554e7ae3cf7e4b3db51012fdc2e4a9ff (127.18.129.1:45293)
I20260807 15:08:21.900853 3516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:60433
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:08:21.925658 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:41555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46291
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:08:22.083842 3859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:08:22.084161 3859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:08:22.084204 3859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:08:22.086146 3859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:08:22.086218 3859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:08:22.086263 3859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:08:22.088954 3859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.129.20:41555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:08:22.089237 3859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:08:22.089556 3859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:08:22.098009 3873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:08:22.098083 3871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:08:22.098340 3870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:08:22.099416 3859 server_base.cc:1034] running on GCE node
I20260807 15:08:22.099781 3859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:08:22.100100 3859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:08:22.101315 3859 hybrid_clock.cc:648] HybridClock initialized: now 1786115302101285 us; error 41 us; skew 500 ppm
Aug 07 15:08:22 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:08:22.106372 3859 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:08:22.106971 3859 webserver.cc:460] Webserver started at http://127.18.129.2:33299/ using document root <none> and password file <none>
I20260807 15:08:22.107262 3859 fs_manager.cc:359] Metadata directory not provided
I20260807 15:08:22.107331 3859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:08:22.107520 3859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:08:22 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
Aug 07 15:08:22 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:08:22.124481 3859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "a761c2c8600a44b5a9f6437307f32f02"
format_stamp: "Formatted at 2026-08-07 15:08:22 on dist-test-slave-5bn2"
server_key: "e8d54166c793806d0d927030e5fd576a"
server_key_iv: "9a242ff331c55b806a3008713d0e5f67"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:22.125001 3859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "a761c2c8600a44b5a9f6437307f32f02"
format_stamp: "Formatted at 2026-08-07 15:08:22 on dist-test-slave-5bn2"
server_key: "e8d54166c793806d0d927030e5fd576a"
server_key_iv: "9a242ff331c55b806a3008713d0e5f67"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:22.146893 3859 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.001s sys 0.001s
I20260807 15:08:22.163118 3881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:22.163520 3859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:08:22.163690 3859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "a761c2c8600a44b5a9f6437307f32f02"
format_stamp: "Formatted at 2026-08-07 15:08:22 on dist-test-slave-5bn2"
server_key: "e8d54166c793806d0d927030e5fd576a"
server_key_iv: "9a242ff331c55b806a3008713d0e5f67"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:22.163898 3859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:08:22.179127 3859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:08:22.181958 3859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:08:22.182154 3859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:08:22.182582 3859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:08:22.183146 3859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:08:22.183260 3859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:22.183359 3859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:08:22.183436 3859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:22.202992 3859 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:32917
I20260807 15:08:22.203444 3859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:08:22.204782 4039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:32917 every 8 connection(s)
I20260807 15:08:22.206223 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 3859
I20260807 15:08:22.206341 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:08:22.206552 18948 external_mini_cluster.cc:1428] Setting key c2ff6b4cedb9aa4727b85a1acfd77d40
Aug 07 15:08:22 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:08:22.218879 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.204416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46291 (local address 127.18.129.2:42813)
0807 15:08:22.204797 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:22.204850 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:08:22.206795 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:08:22.207228 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:08:22.207233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:08:22.207713 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:08:22.208224 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.208231 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.209256 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.209257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.210909 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.210913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.211463 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.211465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.211681 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.212424 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:08:22.212434 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:08:22.214674 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:08:22.216590 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:08:22.216598 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:08:22.216600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:08:22.216893 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:08:22.217611 (+ 718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:08:22.217612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:08:22.217613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:08:22.217761 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:08:22.218081 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:08:22.218089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:08:22.218259 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 15:08:22.218370 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:08:22.218546 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":204,"threads_started":1}
I20260807 15:08:22.219489 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.204414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:42813 (local address 127.18.129.62:46291)
0807 15:08:22.204470 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.204473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.204972 (+ 499us) server_negotiation.cc:407] Connection header received
0807 15:08:22.206958 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.206961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.206979 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.207011 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:08:22.208335 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.209121 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.211006 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.211313 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.211357 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.214788 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:08:22.214790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:08:22.214790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:08:22.214798 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:08:22.216499 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:22.216973 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:22.216975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:22.216975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:22.217034 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:22.217843 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:22.217844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:22.217845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:22.217988 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:08:22.218052 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.219257 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.219361 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:08:22.220302 4040 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46291
I20260807 15:08:22.220405 4040 heartbeater.cc:464] Registering TS with master...
I20260807 15:08:22.220628 4040 heartbeater.cc:511] Master 127.18.129.62:46291 requested a full tablet report, sending...
I20260807 15:08:22.221017 3516 ts_manager.cc:194] Registered new tserver with Master: a761c2c8600a44b5a9f6437307f32f02 (127.18.129.2:32917)
I20260807 15:08:22.221508 3516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:42813
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:08:22.257925 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:41555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46291
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:08:22.428095 4048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:08:22.428319 4048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:08:22.428356 4048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:08:22.429648 4048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:08:22.429733 4048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:08:22.429790 4048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:08:22.431597 4048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.129.20:41555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:08:22.431852 4048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:08:22.432121 4048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:08:22.439397 4055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:08:22.439508 4058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:08:22.439769 4056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:08:22.439796 4048 server_base.cc:1034] running on GCE node
I20260807 15:08:22.440019 4048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:08:22.440205 4048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:08:22.443450 4048 hybrid_clock.cc:648] HybridClock initialized: now 1786115302443428 us; error 45 us; skew 500 ppm
Aug 07 15:08:22 dist-test-slave-5bn2 krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:08:22.447525 4048 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:08:22.448017 4048 webserver.cc:460] Webserver started at http://127.18.129.3:33581/ using document root <none> and password file <none>
I20260807 15:08:22.448262 4048 fs_manager.cc:359] Metadata directory not provided
I20260807 15:08:22.448314 4048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:08:22.448450 4048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:08:22 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:08:22.461465 4048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "6a491b29b38340b8bd1fbaaa40747a99"
format_stamp: "Formatted at 2026-08-07 15:08:22 on dist-test-slave-5bn2"
server_key: "fbaf6f61ae1e848a35f7ac8a9dce30cd"
server_key_iv: "d8a4900438550914698d71898cf30128"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:22.461910 4048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "6a491b29b38340b8bd1fbaaa40747a99"
format_stamp: "Formatted at 2026-08-07 15:08:22 on dist-test-slave-5bn2"
server_key: "fbaf6f61ae1e848a35f7ac8a9dce30cd"
server_key_iv: "d8a4900438550914698d71898cf30128"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:22.464102 4048 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:08:22.478310 4071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:22.478660 4048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:08:22.478926 4048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "6a491b29b38340b8bd1fbaaa40747a99"
format_stamp: "Formatted at 2026-08-07 15:08:22 on dist-test-slave-5bn2"
server_key: "fbaf6f61ae1e848a35f7ac8a9dce30cd"
server_key_iv: "d8a4900438550914698d71898cf30128"
server_key_version: "kuduclusterkey@0"
I20260807 15:08:22.479142 4048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:08:22.526930 4048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:08:22.529441 4048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:08:22.529588 4048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:08:22.530045 4048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:08:22.530553 4048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:08:22.530623 4048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:22.530678 4048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:08:22.530712 4048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:08:22.549785 4048 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:35299
I20260807 15:08:22.550453 4048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:08:22.555936 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 4048
I20260807 15:08:22.556068 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:08:22.556285 18948 external_mini_cluster.cc:1428] Setting key d185454b8434aea01fdd86a0b7e41ae7
I20260807 15:08:22.562444 4238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:35299 every 8 connection(s)
Aug 07 15:08:22 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:08:22.577483 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.566917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:33253 (local address 127.18.129.62:46291)
0807 15:08:22.566966 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.566968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.566978 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:08:22.567006 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.567007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.567018 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.567076 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:08:22.568301 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.569084 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.570918 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.571190 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.571245 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.574385 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:08:22.574387 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:08:22.574387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:08:22.574396 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:08:22.575947 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:22.576422 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:22.576423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:22.576424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:22.576485 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:22.576759 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:22.576760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:22.576760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:22.576888 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:08:22.576928 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.577291 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.577386 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:08:22.578037 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.563429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46291 (local address 127.18.129.3:33253)
0807 15:08:22.564985 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:22.565027 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:08:22.565934 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:08:22.567222 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:08:22.567227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:08:22.567777 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:08:22.568182 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.568191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.569269 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.569270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.570791 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.570794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.571337 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.571338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.571546 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.572311 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:08:22.572322 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:08:22.574262 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:08:22.576041 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:08:22.576047 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:08:22.576049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:08:22.576324 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:08:22.576548 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:08:22.576548 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:08:22.576549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:08:22.576694 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:08:22.576972 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:08:22.576978 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:08:22.577149 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 15:08:22.577624 (+ 475us) client_negotiation.cc:240] Negotiation successful
0807 15:08:22.577803 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1487,"thread_start_us":200,"threads_started":1}
I20260807 15:08:22.578658 4239 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46291
I20260807 15:08:22.578778 4239 heartbeater.cc:464] Registering TS with master...
I20260807 15:08:22.579025 4239 heartbeater.cc:511] Master 127.18.129.62:46291 requested a full tablet report, sending...
I20260807 15:08:22.579459 3516 ts_manager.cc:194] Registered new tserver with Master: 6a491b29b38340b8bd1fbaaa40747a99 (127.18.129.3:35299)
I20260807 15:08:22.579921 3516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:33253
I20260807 15:08:22.582267 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:08:22.592036 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.584123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.18.129.62:46291)
0807 15:08:22.584161 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.584163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.584630 (+ 467us) server_negotiation.cc:407] Connection header received
0807 15:08:22.584690 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.584691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.584701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.584752 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:08:22.585541 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.586253 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.587310 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.587600 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.587671 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.589279 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:08:22.589281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:08:22.589281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:08:22.589289 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:08:22.590842 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:22.591240 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:22.591241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:22.591242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:22.591297 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:08:22.591485 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:08:22.591486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:08:22.591486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:08:22.591602 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:08:22.591671 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.591863 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.591943 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:08:22.594332 3516 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:08:22.596727 3516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:08:22.606696 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.600492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:32917 (local address 127.0.0.1:54858)
0807 15:08:22.600928 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:22.600938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:08:22.600984 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:08:22.601859 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:08:22.601863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:08:22.601867 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:08:22.601938 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.601940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.603486 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.603487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.604850 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.604853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.606338 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.606339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.606426 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.606442 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 15:08:22.606543 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:08:22.606612 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":121,"threads_started":1}
I20260807 15:08:22.607473 4262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.601245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54858 (local address 127.18.129.2:32917)
0807 15:08:22.601605 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.601611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.601622 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:08:22.601688 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.601690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.601706 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.602018 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:08:22.602152 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.603389 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.605130 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.606232 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.607068 (+ 836us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.607259 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.607290 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.607333 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":169,"threads_started":1}
I20260807 15:08:22.608498 3952 tablet_service.cc:1467] Processing CreateTablet for tablet 4e22ffc6d87e4944be871851389e4a56 (DEFAULT_TABLE table=test-table [id=85560ca41ea84abd862906f0139aeec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:08:22.608989 3952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e22ffc6d87e4944be871851389e4a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:08:22.612042 4257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.600712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:45293 (local address 127.0.0.1:40358)
0807 15:08:22.602956 (+ 2244us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:22.602967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:08:22.603075 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:08:22.603353 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:08:22.603354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:08:22.603355 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:08:22.604616 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.604619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.606476 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.606477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.607882 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.607885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.611752 (+ 3867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.611753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.611847 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.611853 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:08:22.611914 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:08:22.611959 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2216,"thread_start_us":103,"threads_started":1}
I20260807 15:08:22.612550 4261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.601334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40358 (local address 127.18.129.1:45293)
0807 15:08:22.602300 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.602304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.603135 (+ 831us) server_negotiation.cc:407] Connection header received
0807 15:08:22.603173 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.603175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.603191 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.603279 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:08:22.604694 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.606365 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.610715 (+ 4350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.611577 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.611651 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.611713 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.612410 (+ 697us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.612471 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":930,"thread_start_us":77,"threads_started":1}
I20260807 15:08:22.613305 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.600145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:35299 (local address 127.0.0.1:53238)
0807 15:08:22.600486 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:22.600531 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:08:22.600618 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:08:22.602362 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:08:22.602363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:08:22.602365 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:08:22.602413 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.602414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.608008 (+ 5594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.608008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.609163 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.609166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.613080 (+ 3914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.613081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.613151 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.613156 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:08:22.613189 (+ 33us) client_negotiation.cc:240] Negotiation successful
0807 15:08:22.613232 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":187,"threads_started":1}
I20260807 15:08:22.614078 4263 tablet_bootstrap.cc:492] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Bootstrap starting.
I20260807 15:08:22.614241 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.600892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.18.129.3:35299)
0807 15:08:22.601899 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.601904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.601924 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:08:22.601965 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.601967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.601984 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.602493 (+ 509us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:08:22.602574 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.604465 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.609647 (+ 5182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.610621 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.613637 (+ 3016us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.613848 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.613882 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.613934 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967,"thread_start_us":113,"threads_started":1}
I20260807 15:08:22.615247 3747 tablet_service.cc:1467] Processing CreateTablet for tablet 4e22ffc6d87e4944be871851389e4a56 (DEFAULT_TABLE table=test-table [id=85560ca41ea84abd862906f0139aeec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:08:22.615638 3747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e22ffc6d87e4944be871851389e4a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:08:22.615759 4263 tablet_bootstrap.cc:654] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Neither blocks nor log segments found. Creating new log.
I20260807 15:08:22.616557 4263 log.cc:824] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Log is configured to *not* fsync() on all Append() calls
I20260807 15:08:22.617518 4263 tablet_bootstrap.cc:492] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: No bootstrap required, opened a new log
I20260807 15:08:22.617580 4263 ts_tablet_manager.cc:1397] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 15:08:22.621336 4263 raft_consensus.cc:348] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.621654 4263 raft_consensus.cc:374] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:08:22.621748 4263 raft_consensus.cc:729] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a761c2c8600a44b5a9f6437307f32f02, State: Initialized, Role: FOLLOWER
I20260807 15:08:22.621949 4263 consensus_queue.cc:260] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.622143 4139 tablet_service.cc:1467] Processing CreateTablet for tablet 4e22ffc6d87e4944be871851389e4a56 (DEFAULT_TABLE table=test-table [id=85560ca41ea84abd862906f0139aeec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:08:22.622402 4263 ts_tablet_manager.cc:1428] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:08:22.622529 4040 heartbeater.cc:503] Master 127.18.129.62:46291 was elected leader, sending a full tablet report...
I20260807 15:08:22.622537 4139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e22ffc6d87e4944be871851389e4a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:08:22.623557 4265 tablet_bootstrap.cc:492] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff: Bootstrap starting.
I20260807 15:08:22.625277 4265 tablet_bootstrap.cc:654] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff: Neither blocks nor log segments found. Creating new log.
I20260807 15:08:22.625969 4265 log.cc:824] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff: Log is configured to *not* fsync() on all Append() calls
I20260807 15:08:22.627377 4265 tablet_bootstrap.cc:492] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff: No bootstrap required, opened a new log
I20260807 15:08:22.627526 4265 ts_tablet_manager.cc:1397] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 15:08:22.631150 4269 tablet_bootstrap.cc:492] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Bootstrap starting.
I20260807 15:08:22.632336 4265 raft_consensus.cc:348] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.632596 4265 raft_consensus.cc:374] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:08:22.632671 4265 raft_consensus.cc:729] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 554e7ae3cf7e4b3db51012fdc2e4a9ff, State: Initialized, Role: FOLLOWER
I20260807 15:08:22.632861 4265 consensus_queue.cc:260] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.633100 4269 tablet_bootstrap.cc:654] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Neither blocks nor log segments found. Creating new log.
I20260807 15:08:22.633258 4265 ts_tablet_manager.cc:1428] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:08:22.633425 3846 heartbeater.cc:503] Master 127.18.129.62:46291 was elected leader, sending a full tablet report...
I20260807 15:08:22.634614 4269 log.cc:824] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Log is configured to *not* fsync() on all Append() calls
I20260807 15:08:22.635885 4269 tablet_bootstrap.cc:492] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: No bootstrap required, opened a new log
I20260807 15:08:22.635977 4269 ts_tablet_manager.cc:1397] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
W20260807 15:08:22.637598 3847 tablet.cc:2369] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:08:22.639819 4269 raft_consensus.cc:348] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.640106 4269 raft_consensus.cc:374] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:08:22.640209 4269 raft_consensus.cc:729] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a491b29b38340b8bd1fbaaa40747a99, State: Initialized, Role: FOLLOWER
I20260807 15:08:22.640368 4269 consensus_queue.cc:260] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.640739 4269 ts_tablet_manager.cc:1428] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260807 15:08:22.640842 4239 heartbeater.cc:503] Master 127.18.129.62:46291 was elected leader, sending a full tablet report...
W20260807 15:08:22.704787 4041 tablet.cc:2369] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 15:08:22 dist-test-slave-5bn2 krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115302, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
W20260807 15:08:22.801604 4240 tablet.cc:2369] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:08:22.830310 4272 raft_consensus.cc:482] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:08:22.830700 4272 raft_consensus.cc:504] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.831372 4272 leader_election.cc:288] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a761c2c8600a44b5a9f6437307f32f02 (127.18.129.2:32917), 554e7ae3cf7e4b3db51012fdc2e4a9ff (127.18.129.1:45293)
I20260807 15:08:22.836300 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.831681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:32917 (local address 127.18.129.3:33483)
0807 15:08:22.832150 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:22.832163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:08:22.832214 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:08:22.832933 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:08:22.832935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:08:22.832937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:08:22.833099 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.833102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.833973 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.833974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.835177 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.835181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.835990 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.835991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.836130 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.836134 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:08:22.836184 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:08:22.836222 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445}
I20260807 15:08:22.836947 4262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.832360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:33483 (local address 127.18.129.2:32917)
0807 15:08:22.832481 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.832483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.832491 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:08:22.832520 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.832521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.832532 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.832576 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:08:22.833195 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.833856 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.835332 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.835879 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.835950 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.835960 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.836835 (+ 875us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.836880 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 15:08:22.837236 3974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e22ffc6d87e4944be871851389e4a56" candidate_uuid: "6a491b29b38340b8bd1fbaaa40747a99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a761c2c8600a44b5a9f6437307f32f02" is_pre_election: true
I20260807 15:08:22.837415 3974 raft_consensus.cc:2454] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a491b29b38340b8bd1fbaaa40747a99 in term 0.
I20260807 15:08:22.838017 4073 leader_election.cc:302] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a491b29b38340b8bd1fbaaa40747a99, a761c2c8600a44b5a9f6437307f32f02; no voters:
I20260807 15:08:22.838181 4272 raft_consensus.cc:2781] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:08:22.838198 4261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.832351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:51835 (local address 127.18.129.1:45293)
0807 15:08:22.832411 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.832413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.834151 (+ 1738us) server_negotiation.cc:407] Connection header received
0807 15:08:22.834247 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.834248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.834258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.834287 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:08:22.834695 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.835381 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.837005 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.837644 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.837695 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.837704 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.838088 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.838135 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:08:22.838225 4272 raft_consensus.cc:482] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:08:22.838248 4272 raft_consensus.cc:3037] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:08:22.838467 4273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.831780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:45293 (local address 127.18.129.3:51835)
0807 15:08:22.834056 (+ 2276us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:22.834066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:08:22.834178 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:08:22.834349 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:08:22.834350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:08:22.834351 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:08:22.834605 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.834607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.835465 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.835465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.836906 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:08:22.836909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.837860 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.837861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:08:22.837986 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.837991 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:08:22.838301 (+ 310us) client_negotiation.cc:240] Negotiation successful
0807 15:08:22.838344 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2223,"thread_start_us":152,"threads_started":1}
I20260807 15:08:22.839529 4272 raft_consensus.cc:504] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.839638 4272 leader_election.cc:288] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [CANDIDATE]: Term 1 election: Requested vote from peers a761c2c8600a44b5a9f6437307f32f02 (127.18.129.2:32917), 554e7ae3cf7e4b3db51012fdc2e4a9ff (127.18.129.1:45293)
I20260807 15:08:22.839797 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e22ffc6d87e4944be871851389e4a56" candidate_uuid: "6a491b29b38340b8bd1fbaaa40747a99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" is_pre_election: true
I20260807 15:08:22.839885 3974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e22ffc6d87e4944be871851389e4a56" candidate_uuid: "6a491b29b38340b8bd1fbaaa40747a99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a761c2c8600a44b5a9f6437307f32f02"
I20260807 15:08:22.839946 3772 raft_consensus.cc:2454] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a491b29b38340b8bd1fbaaa40747a99 in term 0.
I20260807 15:08:22.839954 3974 raft_consensus.cc:3037] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:08:22.840088 3771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e22ffc6d87e4944be871851389e4a56" candidate_uuid: "6a491b29b38340b8bd1fbaaa40747a99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff"
I20260807 15:08:22.840207 3771 raft_consensus.cc:3037] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:08:22.841281 3974 raft_consensus.cc:2454] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a491b29b38340b8bd1fbaaa40747a99 in term 1.
I20260807 15:08:22.841521 4073 leader_election.cc:302] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a491b29b38340b8bd1fbaaa40747a99, a761c2c8600a44b5a9f6437307f32f02; no voters:
I20260807 15:08:22.841604 3771 raft_consensus.cc:2454] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a491b29b38340b8bd1fbaaa40747a99 in term 1.
I20260807 15:08:22.842177 4272 raft_consensus.cc:2781] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:08:22.842337 4272 raft_consensus.cc:686] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 1 LEADER]: Becoming Leader. State: Replica: 6a491b29b38340b8bd1fbaaa40747a99, State: Running, Role: LEADER
I20260807 15:08:22.842444 4272 consensus_queue.cc:237] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } }
I20260807 15:08:22.843391 3516 catalog_manager.cc:5495] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a491b29b38340b8bd1fbaaa40747a99 (127.18.129.3). New cstate: current_term: 1 leader_uuid: "6a491b29b38340b8bd1fbaaa40747a99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a491b29b38340b8bd1fbaaa40747a99" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 } health_report { overall_health: UNKNOWN } } }
I20260807 15:08:22.893496 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.890885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.18.129.3:35299)
0807 15:08:22.890948 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:08:22.890951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:08:22.891004 (+ 53us) server_negotiation.cc:407] Connection header received
0807 15:08:22.891089 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:08:22.891091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:08:22.891111 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:08:22.891174 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:08:22.891312 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.891828 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.892598 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:08:22.892845 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:08:22.892901 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:08:22.893110 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:08:22.893255 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:08:22.893338 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:08:22.893388 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 15:08:22.893413 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:08:22.902809 3771 raft_consensus.cc:1261] T 4e22ffc6d87e4944be871851389e4a56 P 554e7ae3cf7e4b3db51012fdc2e4a9ff [term 1 FOLLOWER]: Refusing update from remote peer 6a491b29b38340b8bd1fbaaa40747a99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:08:22.902922 3974 raft_consensus.cc:1261] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 1 FOLLOWER]: Refusing update from remote peer 6a491b29b38340b8bd1fbaaa40747a99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:08:22.903302 4272 consensus_queue.cc:1035] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "554e7ae3cf7e4b3db51012fdc2e4a9ff" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:08:22.903528 4275 consensus_queue.cc:1035] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a761c2c8600a44b5a9f6437307f32f02" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 32917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:08:22.909116 4280 mvcc.cc:205] Tried to move back new op lower bound from 7315928280686460928 to 7315928280443146240. Current Snapshot: MvccSnapshot[applied={T|T < 7315928280686460928}]
I20260807 15:08:22.909919 3516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:08:22.910053 3516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:08:22.911106 3516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93aa4a1cf13d46aeb954c4c7fb5999d5: Sending DeleteTablet for 3 replicas of tablet 4e22ffc6d87e4944be871851389e4a56
I20260807 15:08:22.911746 4139 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e22ffc6d87e4944be871851389e4a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:08:22 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:53238
I20260807 15:08:22.911892 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 3646
I20260807 15:08:22.912057 3952 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e22ffc6d87e4944be871851389e4a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:08:22 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:54858
I20260807 15:08:22.938951 4290 tablet_replica.cc:330] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: stopping tablet replica
I20260807 15:08:22.939110 4290 raft_consensus.cc:2229] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:08:22.939224 4290 raft_consensus.cc:2258] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:08:22.939193 4281 mvcc.cc:205] Tried to move back new op lower bound from 7315928280686460928 to 7315928280443146240. Current Snapshot: MvccSnapshot[applied={T|T < 7315928280686460928}]
I20260807 15:08:22.939721 4290 ts_tablet_manager.cc:1905] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:08:22.942265 4290 ts_tablet_manager.cc:1918] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:08:22.942420 4290 log.cc:1196] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/4e22ffc6d87e4944be871851389e4a56
I20260807 15:08:22.942781 4290 ts_tablet_manager.cc:1939] T 4e22ffc6d87e4944be871851389e4a56 P 6a491b29b38340b8bd1fbaaa40747a99: Deleting consensus metadata
I20260807 15:08:22.943287 3495 catalog_manager.cc:4841] TS 6a491b29b38340b8bd1fbaaa40747a99 (127.18.129.3:35299): tablet 4e22ffc6d87e4944be871851389e4a56 (table test-table [id=85560ca41ea84abd862906f0139aeec0]) successfully deleted
I20260807 15:08:22.948580 4291 tablet_replica.cc:330] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: stopping tablet replica
I20260807 15:08:22.948797 4291 raft_consensus.cc:2229] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:08:22.948968 4291 raft_consensus.cc:2258] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:08:22.949579 4291 ts_tablet_manager.cc:1905] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:08:22.951647 4291 ts_tablet_manager.cc:1918] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:08:22.951715 4291 log.cc:1196] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/4e22ffc6d87e4944be871851389e4a56
I20260807 15:08:22.951959 4291 ts_tablet_manager.cc:1939] T 4e22ffc6d87e4944be871851389e4a56 P a761c2c8600a44b5a9f6437307f32f02: Deleting consensus metadata
I20260807 15:08:22.952409 3495 catalog_manager.cc:4841] TS a761c2c8600a44b5a9f6437307f32f02 (127.18.129.2:32917): tablet 4e22ffc6d87e4944be871851389e4a56 (table test-table [id=85560ca41ea84abd862906f0139aeec0]) successfully deleted
W20260807 15:08:22.952781 3495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:45293 (error 108)
I20260807 15:08:22.953779 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:08:22.953623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:45293 (local address 127.0.0.1:40366)
0807 15:08:22.953681 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:08:22.953741 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:45293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:08:22.953781 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 3859
W20260807 15:08:22.963806 3495 catalog_manager.cc:4584] TS 554e7ae3cf7e4b3db51012fdc2e4a9ff (127.18.129.1:45293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e22ffc6d87e4944be871851389e4a56: Network error: Client connection negotiation failed: client connection to 127.18.129.1:45293: connect: Connection refused (error 111)
I20260807 15:08:23.001062 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 4048
I20260807 15:08:23.012903 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 3467
I20260807 15:08:23.022774 18948 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:08:23.366747 18948 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:08:23.366894 18948 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:08:23.710988 18948 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:08:23.711 UTC [2833] LOG: received smart shutdown request
2026-08-07 15:08:23.715 UTC [2833] LOG: background worker "logical replication launcher" (PID 2848) exited with exit code 1
2026-08-07 15:08:23.715 UTC [2839] LOG: shutting down
2026-08-07 15:08:23.872 UTC [2833] LOG: database system is shut down
2026-08-07T15:08:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50764 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:08:24 dist-test-slave-5bn2 krb5kdc[4306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:08:24 dist-test-slave-5bn2 krb5kdc[4306](info): set up 2 sockets
Aug 07 15:08:24 dist-test-slave-5bn2 krb5kdc[4306](info): commencing operation
krb5kdc: starting...
W20260807 15:08:25.119418 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:08:25 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:08:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:08:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.129.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.129.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/rangeradmin_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/rangeradmin_127.18.129.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.129.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.129.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/rangerlookup_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/rangerlookup_127.18.129.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.129.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.129.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/HTTP_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/HTTP_127.18.129.20@KRBTEST.COM.keytab.
I20260807 15:08:25.278556 18948 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDKyZDM/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/postgres -l logfile start
2026-08-07 15:08:27.591 UTC [4354] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:08:27.591 UTC [4354] LOG: listening on IPv4 address "127.18.129.20", port 48411
2026-08-07 15:08:27.595 UTC [4354] LOG: listening on Unix socket "/tmp/.s.PGSQL.48411"
2026-08-07 15:08:27.616 UTC [4358] LOG: database system was shut down at 2026-08-07 15:08:26 UTC
2026-08-07 15:08:27.619 UTC [4354] LOG: database system is ready to accept connections
I20260807 15:08:28.689425 18948 mini_postgres.cc:96] Postgres bound to 48411
2026-08-07 15:08:28.696 UTC [4369] FATAL: database "slave" does not exist
127.18.129.20:48411 - accepting connections
I20260807 15:08:28.697070 18948 mini_ranger.cc:161] Starting Ranger...
I20260807 15:08:28.712110 18948 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:08:29.141993 18948 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:08:29.142092 18948 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-admin
2026-08-07 15:08:29,420 [I] DB FLAVOR :POSTGRES
2026-08-07 15:08:29,420 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:08:29,421 [I] Checking connection..
2026-08-07 15:08:29,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:08:29,654 [I] Checking connection passed.
2026-08-07 15:08:29,654 [I] --------- Verifying version history table ---------
2026-08-07 15:08:29,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:08:29,918 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:08:29,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:08:30,181 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:08:30,182 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:08:30,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:08:30.450 UTC [4467] WARNING: there is no transaction in progress
2026-08-07 15:08:30,457 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:08:30,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:08:30,697 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:08:30,697 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:08:30,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:08:30,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:08:31,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5bn2.c.gcp-upstream.internal','N') ;"
2026-08-07 15:08:31,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:08:31,707 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:08:31,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:08:31,954 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:08:31,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:08:32,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:08:32,476 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:08:32,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:08:33.402 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.418 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.434 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.448 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.463 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.481 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.537 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.544 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.558 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.568 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.578 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.591 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.600 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.607 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.613 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:33.619 UTC [4630] WARNING: there is no transaction in progress
2026-08-07 15:08:34,230 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:08:34,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:08:34,465 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:08:34,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:08:34,729 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:08:34,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:08:34,977 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:08:34,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:08:35,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:08:35,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5bn2.c.gcp-upstream.internal';"
2026-08-07 15:08:35,740 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:08:35,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:08:35,995 [I] DB_PATCHES have already been applied
I20260807 15:08:36.000859 18948 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDKyZDM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:08:36.000924 18948 mini_ranger.cc:193] Using host: 127.18.129.20
I20260807 15:08:36.004577 18948 mini_ranger.cc:227] Ranger admin URL: http://127.18.129.20:50893
Aug 07, 2026 3:08:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:08:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:08:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:08:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:08:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:08:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:08:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:08:38.530922 18948 mini_ranger.cc:160] Time spent starting Ranger: real 9.834s user 0.001s sys 0.011s
Aug 07, 2026 3:08:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:08:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 3:08:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:08:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:08:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:09:00.160027 18948 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.129.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.129.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/rangerkms_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.129.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/rangerkms_127.18.129.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.129.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.129.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.129.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/HTTP_127.18.129.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.129.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/HTTP_127.18.129.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:09:00 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115340, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:09:00.244243 18948 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:09:00.254729 18948 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:09:01.078436 18948 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:09:01.078513 18948 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms
I20260807 15:09:01.078526 18948 mini_ranger_kms.cc:227] Using postgres at 127.18.129.20:48411
2026-08-07 15:09:01,258 [I] DB FLAVOR :POSTGRES
2026-08-07 15:09:01,258 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:09:01,258 [I] Checking connection
2026-08-07 15:09:01,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:09:01,507 [I] connection success
2026-08-07 15:09:01,507 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:09:01,507 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:09:01,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:09:01,769 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:09:01,770 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:09:01,770 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:09:01,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.129.20:48411/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:09:02,051 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:09:02.494477 18948 mini_ranger_kms.cc:310] Created kms service
I20260807 15:09:02.718197 18948 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:09:02.764765 18948 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:09:02 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115340, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:03.169411 18948 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:09:03.169596 18948 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDKyZDM/build/release/bin/postgresql.jar:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDKyZDM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:09:03.169615 18948 mini_ranger_kms.cc:240] Using host: 127.18.129.20
I20260807 15:09:03.174606 18948 mini_ranger_kms.cc:276] Ranger KMS PID: 4913
I20260807 15:09:03.174683 18948 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.129.20:37959
Aug 07, 2026 3:09:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:09:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:09:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:09:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:09:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:09:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:09:05 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115345, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.129.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:05.610148 18948 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.366s user 0.004s sys 0.157s
I20260807 15:09:05.610262 18948 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:09:05.610311 18948 mini_ranger_kms.cc:401] 127.18.129.20:37959/kms/v1/keys
Aug 07 15:09:05 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115345, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.129.20@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
Aug 07 15:09:06 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115345, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.129.20@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
Aug 07 15:09:08 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:09:08.647030 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46587
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46587
--ranger_config_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:37959/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:08.753299 4988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:08.753507 4988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:08.753528 4988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:08.754683 4988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:08.754726 4988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:08.754739 4988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:08.754750 4988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:08.754760 4988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:08.756333 4988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.129.20:37959/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46587
--ranger_config_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:08.756572 4988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:08.756796 4988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:08.762117 4997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:08.762243 4988 server_base.cc:1034] running on GCE node
W20260807 15:09:08.762128 4998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:08.762130 5000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:08.762961 4988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:08.763301 4988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:08.764520 4988 hybrid_clock.cc:648] HybridClock initialized: now 1786115348764496 us; error 41 us; skew 500 ppm
Aug 07 15:09:08 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115348, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:08.772033 4988 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:08.772593 4988 webserver.cc:460] Webserver started at http://127.18.129.62:33529/ using document root <none> and password file <none>
I20260807 15:09:08.772837 4988 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:08.772888 4988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:08.773072 4988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:09:08 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115348, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:08.821099 4988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "83c7089f938340a3b284d5acb8b83428"
format_stamp: "Formatted at 2026-08-07 15:09:08 on dist-test-slave-5bn2"
server_key: "3e1364bceac2c320fc2ef43a6f85a051"
server_key_iv: "85e2f57aa19abd016effddb4b999d8e9"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:08.821491 4988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "83c7089f938340a3b284d5acb8b83428"
format_stamp: "Formatted at 2026-08-07 15:09:08 on dist-test-slave-5bn2"
server_key: "3e1364bceac2c320fc2ef43a6f85a051"
server_key_iv: "85e2f57aa19abd016effddb4b999d8e9"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:08.822906 4988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:09:08.837987 5009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:08.838457 4988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:08.838575 4988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "83c7089f938340a3b284d5acb8b83428"
format_stamp: "Formatted at 2026-08-07 15:09:08 on dist-test-slave-5bn2"
server_key: "3e1364bceac2c320fc2ef43a6f85a051"
server_key_iv: "85e2f57aa19abd016effddb4b999d8e9"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:08.838675 4988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:08.860468 4988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:08.862458 4988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:08.862630 4988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:08.867861 4988 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:46587
I20260807 15:09:08.867895 5078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:46587 every 8 connection(s)
I20260807 15:09:08.868264 4988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:08.868985 5079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:08.871471 5079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Bootstrap starting.
I20260807 15:09:08.872097 5079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:08.872365 5079 log.cc:824] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:08.873064 5079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: No bootstrap required, opened a new log
I20260807 15:09:08.874456 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 4988
I20260807 15:09:08.874562 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:09:08.874737 18948 external_mini_cluster.cc:1428] Setting key 14394e96c0e8e90ad604de1045af8a7b
I20260807 15:09:08.875571 5079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } }
I20260807 15:09:08.875732 5079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:08.875767 5079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c7089f938340a3b284d5acb8b83428, State: Initialized, Role: FOLLOWER
I20260807 15:09:08.875931 5079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } }
I20260807 15:09:08.876000 5079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:08.876031 5079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:08.876094 5079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:08.876837 5079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } }
I20260807 15:09:08.876948 5079 leader_election.cc:302] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83c7089f938340a3b284d5acb8b83428; no voters:
I20260807 15:09:08.877147 5079 leader_election.cc:288] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:08.877234 5086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:08.877422 5079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:08.877430 5086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 1 LEADER]: Becoming Leader. State: Replica: 83c7089f938340a3b284d5acb8b83428, State: Running, Role: LEADER
I20260807 15:09:08.877620 5086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } }
I20260807 15:09:08.877912 5086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83c7089f938340a3b284d5acb8b83428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } } }
I20260807 15:09:08.877969 5086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:08.878068 5086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83c7089f938340a3b284d5acb8b83428. Latest consensus state: current_term: 1 leader_uuid: "83c7089f938340a3b284d5acb8b83428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } } }
I20260807 15:09:08.878103 5086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:08.878350 5100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:08.878911 5100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:08.879251 5079 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 07 15:09:08 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:08.885654 5084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:08.875343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.18.129.62:46587)
0807 15:09:08.875519 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:08.875528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:08.875559 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:09:08.875701 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:08.875708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:08.875764 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:08.875907 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:08.876801 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:08.878309 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:08.879592 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:08.880148 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:08.880248 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:08.882578 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:08.882582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:08.882582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:08.882592 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:08.884089 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:08.884417 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:08.884420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:08.884420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:08.884519 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:08.884694 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:08.884695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:08.884695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:08.884947 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:08.885149 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:08.885218 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 15:09:08.885439 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260807 15:09:08.891708 5111 mvcc.cc:205] Tried to move back new op lower bound from 7315928469012127744 to 7315928469003005952. Current Snapshot: MvccSnapshot[applied={T|T < 7315928469012127744}]
I20260807 15:09:08.892587 5100 catalog_manager.cc:1347] Generated new cluster ID: 592bd9c1dd1f40e2814f76ee20f87d15
I20260807 15:09:08.892642 5100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:08.914290 5100 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:08.915163 5100 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:08.926643 5100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Generated new TSK 0
I20260807 15:09:08.926963 5100 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:09:09.658103 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:37959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46587
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:09.763212 5146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:09.763464 5146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:09.763494 5146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:09.764775 5146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:09.764820 5146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:09.764846 5146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:09:09.766331 5146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.129.20:37959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:09.766582 5146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:09.766800 5146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:09.772567 5156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:09.772725 5146 server_base.cc:1034] running on GCE node
W20260807 15:09:09.772544 5155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:09.772799 5158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:09.773056 5146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:09.773278 5146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:09.774453 5146 hybrid_clock.cc:648] HybridClock initialized: now 1786115349774434 us; error 39 us; skew 500 ppm
Aug 07 15:09:09 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115349, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:09.777070 5146 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:09:09.777520 5146 webserver.cc:460] Webserver started at http://127.18.129.1:35089/ using document root <none> and password file <none>
I20260807 15:09:09.777745 5146 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:09.777794 5146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:09.777947 5146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:09:09 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115349, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:09.793540 5146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "e455a293eb3849b7b483d28992e6e99f"
format_stamp: "Formatted at 2026-08-07 15:09:09 on dist-test-slave-5bn2"
server_key: "aa961580906f0915d3b6269c81439161"
server_key_iv: "bb647de65c08b42d0d48b2f77d8ac628"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:09.794094 5146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "e455a293eb3849b7b483d28992e6e99f"
format_stamp: "Formatted at 2026-08-07 15:09:09 on dist-test-slave-5bn2"
server_key: "aa961580906f0915d3b6269c81439161"
server_key_iv: "bb647de65c08b42d0d48b2f77d8ac628"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:09.795804 5146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:09.825717 5170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:09.825992 5146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:09:09.826093 5146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "e455a293eb3849b7b483d28992e6e99f"
format_stamp: "Formatted at 2026-08-07 15:09:09 on dist-test-slave-5bn2"
server_key: "aa961580906f0915d3b6269c81439161"
server_key_iv: "bb647de65c08b42d0d48b2f77d8ac628"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:09.826192 5146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:09.856312 5146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:09.858382 5146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:09.858639 5146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:09.858971 5146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:09.859409 5146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:09.859513 5146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:09.859642 5146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:09.859727 5146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:09.866127 5146 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:36357
I20260807 15:09:09.866644 5146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:09.872191 5327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:36357 every 8 connection(s)
I20260807 15:09:09.876226 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 5146
I20260807 15:09:09.876328 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:09.876492 18948 external_mini_cluster.cc:1428] Setting key 80bc3faaba45233ff99c0cb6ab69bb4b
Aug 07 15:09:09 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115349, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:09.885528 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:09.872778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:45455 (local address 127.18.129.62:46587)
0807 15:09:09.872920 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:09.872922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:09.875835 (+ 2913us) server_negotiation.cc:407] Connection header received
0807 15:09:09.875864 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:09.875865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:09.875879 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:09.875937 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:09.876734 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:09.877452 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:09.878783 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:09.879084 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:09.879129 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:09.881684 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:09.881685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:09.881686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:09.881693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:09.883930 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:09.884396 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:09.884397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:09.884398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:09.884465 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:09.884762 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:09.884763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:09.884763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:09.884904 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:09.884945 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:09.885325 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 15:09:09.885419 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260807 15:09:09.886324 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:09.872779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46587 (local address 127.18.129.1:45455)
0807 15:09:09.874999 (+ 2220us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:09.875054 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:09.875697 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:09.876068 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:09.876073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:09.876315 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:09.876620 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:09.876626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:09.877576 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:09.877576 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:09.878683 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:09.878686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:09.879238 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:09.879239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:09.879385 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:09.879920 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:09.879931 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:09.881578 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:09.884022 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:09.884029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:09.884031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:09.884313 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:09.884529 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:09.884530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:09.884530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:09.884687 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:09.884986 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:09.884992 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:09.885159 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 15:09:09.885885 (+ 726us) client_negotiation.cc:240] Negotiation successful
0807 15:09:09.886053 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2173,"thread_start_us":74,"threads_started":1}
I20260807 15:09:09.887450 5329 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46587
I20260807 15:09:09.887552 5329 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:09.887741 5329 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:09.888306 5032 ts_manager.cc:194] Registered new tserver with Master: e455a293eb3849b7b483d28992e6e99f (127.18.129.1:36357)
I20260807 15:09:09.889201 5032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:45455
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:09:09.918141 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:37959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46587
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:09:09 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115349, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 15:09:10.009995 5347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:10.010254 5347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:10.010288 5347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:10.011555 5347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:10.011607 5347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:10.011644 5347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:09:10.013816 5347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.129.20:37959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:10.014084 5347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:10.014359 5347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:10.021642 5355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.022248 5357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.022773 5359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:10.022970 5347 server_base.cc:1034] running on GCE node
I20260807 15:09:10.023201 5347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:10.023487 5347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:10.024616 5347 hybrid_clock.cc:648] HybridClock initialized: now 1786115350024595 us; error 35 us; skew 500 ppm
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:10.028954 5347 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:09:10.029500 5347 webserver.cc:460] Webserver started at http://127.18.129.2:41127/ using document root <none> and password file <none>
I20260807 15:09:10.029779 5347 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:10.029829 5347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:10.030018 5347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:10.044744 5347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "2e8ed9a256094603ac3c1473058437ed"
format_stamp: "Formatted at 2026-08-07 15:09:10 on dist-test-slave-5bn2"
server_key: "d3ce3ea922af6a26f71baf2cc5156c30"
server_key_iv: "f52239c4ab549adde76ed6ecd34f811a"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.045186 5347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "2e8ed9a256094603ac3c1473058437ed"
format_stamp: "Formatted at 2026-08-07 15:09:10 on dist-test-slave-5bn2"
server_key: "d3ce3ea922af6a26f71baf2cc5156c30"
server_key_iv: "f52239c4ab549adde76ed6ecd34f811a"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.054293 5347 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.000s
I20260807 15:09:10.067413 5375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.067672 5347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:10.067759 5347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "2e8ed9a256094603ac3c1473058437ed"
format_stamp: "Formatted at 2026-08-07 15:09:10 on dist-test-slave-5bn2"
server_key: "d3ce3ea922af6a26f71baf2cc5156c30"
server_key_iv: "f52239c4ab549adde76ed6ecd34f811a"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.067852 5347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:10.093729 5347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:10.095808 5347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:10.095968 5347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:10.096215 5347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:10.096573 5347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:10.096613 5347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.096645 5347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:10.096666 5347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.103636 5347 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:34019
I20260807 15:09:10.103729 5547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:34019 every 8 connection(s)
I20260807 15:09:10.104033 5347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:10.110709 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 5347
I20260807 15:09:10.110814 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:10.111150 18948 external_mini_cluster.cc:1428] Setting key f9e414830885400cdd318506ef3f461a
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:10.115526 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:10.104805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:41957 (local address 127.18.129.62:46587)
0807 15:09:10.104858 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:10.104860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:10.105238 (+ 378us) server_negotiation.cc:407] Connection header received
0807 15:09:10.105895 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:10.105896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:10.105913 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:10.105996 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:10.106801 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.107639 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.109134 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.109479 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.109521 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:10.112129 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:10.112131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:10.112131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:10.112139 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:10.113798 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:10.114308 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:10.114309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:10.114310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:10.114381 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:10.114706 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:10.114707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:10.114708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:10.114893 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:10.114974 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:10.115306 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 15:09:10.115417 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:09:10.115705 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:10.104826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46587 (local address 127.18.129.2:41957)
0807 15:09:10.105084 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:10.105129 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:10.105778 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:10.106140 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:10.106145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:10.106408 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:10.106684 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:10.106690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.107758 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.107758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:10.109035 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:10.109038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.109621 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.109622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:10.109761 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:10.110386 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:10.110396 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:10.112004 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:10.113891 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:10.113899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:10.113901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:10.114220 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:10.114485 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:10.114486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:10.114486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:10.114628 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:10.114963 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:10.114969 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:10.115143 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 15:09:10.115264 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 15:09:10.115422 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":80,"threads_started":1}
I20260807 15:09:10.116220 5549 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46587
I20260807 15:09:10.116320 5549 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:10.116515 5549 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:10.116866 5032 ts_manager.cc:194] Registered new tserver with Master: 2e8ed9a256094603ac3c1473058437ed (127.18.129.2:34019)
I20260807 15:09:10.117228 5032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:41957
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:09:10.139703 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:37959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46587
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:10.232093 5555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:10.232311 5555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:10.232342 5555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:10.233479 5555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:10.233522 5555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:10.233557 5555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:09:10.235080 5555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.129.20:37959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:46587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:10.235327 5555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:10.235602 5555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:10.241187 5564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.241511 5561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.241168 5562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:10.242519 5555 server_base.cc:1034] running on GCE node
I20260807 15:09:10.242724 5555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:10.243019 5555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:10.244184 5555 hybrid_clock.cc:648] HybridClock initialized: now 1786115350244165 us; error 45 us; skew 500 ppm
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:10.247036 5555 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:09:10.247495 5555 webserver.cc:460] Webserver started at http://127.18.129.3:37083/ using document root <none> and password file <none>
I20260807 15:09:10.247754 5555 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:10.247821 5555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:10.247957 5555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:10.260401 5555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d"
format_stamp: "Formatted at 2026-08-07 15:09:10 on dist-test-slave-5bn2"
server_key: "92bf7c080a2232faa5e1d5d6dad1ce25"
server_key_iv: "c40eddcb752efd0227c16c7d8cec4340"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.260991 5555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d"
format_stamp: "Formatted at 2026-08-07 15:09:10 on dist-test-slave-5bn2"
server_key: "92bf7c080a2232faa5e1d5d6dad1ce25"
server_key_iv: "c40eddcb752efd0227c16c7d8cec4340"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.267037 5555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260807 15:09:10.286373 5578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.286563 5555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260807 15:09:10.286666 5555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d"
format_stamp: "Formatted at 2026-08-07 15:09:10 on dist-test-slave-5bn2"
server_key: "92bf7c080a2232faa5e1d5d6dad1ce25"
server_key_iv: "c40eddcb752efd0227c16c7d8cec4340"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.286767 5555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:10.312134 5555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:10.314698 5555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:10.314893 5555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:10.315150 5555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:10.315546 5555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:10.315598 5555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.315635 5555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:10.315658 5555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.321929 5555 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:44973
I20260807 15:09:10.322367 5555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:10.325497 5749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:44973 every 8 connection(s)
I20260807 15:09:10.325965 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 5555
I20260807 15:09:10.326092 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:10.326283 18948 external_mini_cluster.cc:1428] Setting key b8955622200818d08fcbfffcf0fbe40f
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:10.338219 5335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:10.326957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:38307 (local address 127.18.129.62:46587)
0807 15:09:10.327004 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:10.327006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:10.327400 (+ 394us) server_negotiation.cc:407] Connection header received
0807 15:09:10.328110 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:10.328111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:10.328124 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:10.328153 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:10.329039 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.329840 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.331402 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.331694 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.331743 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:10.334439 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:10.334441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:10.334441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:10.334449 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:10.336175 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:10.337038 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:10.337040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:10.337040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:10.337111 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:10.337429 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:10.337430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:10.337430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:10.337589 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:10.337641 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:10.338028 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 15:09:10.338117 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:09:10.338778 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:10.327022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46587 (local address 127.18.129.3:38307)
0807 15:09:10.327279 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:10.327320 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:10.327988 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:10.328342 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:10.328346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:10.328616 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:10.328919 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:10.328927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.329959 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.329960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:10.331297 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:10.331301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.331855 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.331856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:10.331999 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:10.332644 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:10.332654 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:10.334326 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:10.336617 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:10.336624 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:10.336626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:10.336942 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:10.337188 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:10.337189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:10.337189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:10.337337 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:10.337695 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:10.337702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:10.337872 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 15:09:10.338324 (+ 452us) client_negotiation.cc:240] Negotiation successful
0807 15:09:10.338496 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":81,"threads_started":1}
I20260807 15:09:10.339273 5750 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46587
I20260807 15:09:10.339372 5750 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:10.339565 5750 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:10.339996 5032 ts_manager.cc:194] Registered new tserver with Master: 8fc9dc6315f44fcdbb8745d3e17bca1d (127.18.129.3:44973)
I20260807 15:09:10.340488 5032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:38307
I20260807 15:09:10.348917 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:10.349635 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 5146
I20260807 15:09:10.389012 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 5347
I20260807 15:09:10.396016 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 5555
I20260807 15:09:10.401989 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 4988
I20260807 15:09:10.416530 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46587
--webserver_interface=127.18.129.62
--webserver_port=33529
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46587
--ranger_config_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:37959/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.129.20:37959/kms with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:10.496892 5756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:10.497112 5756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:10.497143 5756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:10.498553 5756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:10.498616 5756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:10.498638 5756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:10.498657 5756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:10.498672 5756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:10.500407 5756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.129.20:37959/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46587
--ranger_config_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=33529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:10.500643 5756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:10.500851 5756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:10.505640 5763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.505667 5762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.505640 5765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:10.505676 5756 server_base.cc:1034] running on GCE node
I20260807 15:09:10.506016 5756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:10.506273 5756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:10.507407 5756 hybrid_clock.cc:648] HybridClock initialized: now 1786115350507389 us; error 33 us; skew 500 ppm
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:10.509785 5756 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:10.510221 5756 webserver.cc:460] Webserver started at http://127.18.129.62:33529/ using document root <none> and password file <none>
I20260807 15:09:10.510430 5756 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:10.510484 5756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:10.585716 5756 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:10.586628 5774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.586827 5756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:10.586982 5756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "83c7089f938340a3b284d5acb8b83428"
format_stamp: "Formatted at 2026-08-07 15:09:08 on dist-test-slave-5bn2"
server_key: "3e1364bceac2c320fc2ef43a6f85a051"
server_key_iv: "85e2f57aa19abd016effddb4b999d8e9"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.587509 5756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:10.599412 5756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:10.601270 5756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:10.601387 5756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:10.605461 5756 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:46587
I20260807 15:09:10.605502 5859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:46587 every 8 connection(s)
I20260807 15:09:10.605872 5756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:10.607183 5861 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:09:10.607823 5861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Bootstrap starting.
I20260807 15:09:10.612550 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 5756
I20260807 15:09:10.612808 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:37959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:36357
--local_ip_for_outbound_sockets=127.18.129.1
--tserver_master_addrs=127.18.129.62:46587
--webserver_port=35089
--webserver_interface=127.18.129.1
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.129.20:37959/kms with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:10.615711 5861 log.cc:824] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:10.617646 5861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:09:10.617928 5861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Bootstrap complete.
I20260807 15:09:10.620260 5861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } }
I20260807 15:09:10.620504 5861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c7089f938340a3b284d5acb8b83428, State: Initialized, Role: FOLLOWER
I20260807 15:09:10.620661 5861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } }
I20260807 15:09:10.620740 5861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:10.620791 5861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:10.620839 5861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:09:10.621856 5861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } }
I20260807 15:09:10.621960 5861 leader_election.cc:302] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83c7089f938340a3b284d5acb8b83428; no voters:
I20260807 15:09:10.622131 5861 leader_election.cc:288] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:09:10.622179 5868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:09:10.622292 5868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [term 2 LEADER]: Becoming Leader. State: Replica: 83c7089f938340a3b284d5acb8b83428, State: Running, Role: LEADER
I20260807 15:09:10.622380 5868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } }
I20260807 15:09:10.622406 5861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:10.622695 5869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "83c7089f938340a3b284d5acb8b83428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } } }
I20260807 15:09:10.622792 5869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:10.622699 5868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83c7089f938340a3b284d5acb8b83428. Latest consensus state: current_term: 2 leader_uuid: "83c7089f938340a3b284d5acb8b83428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83c7089f938340a3b284d5acb8b83428" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46587 } } }
I20260807 15:09:10.623013 5868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:10.623122 5874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:10.623847 5874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:10.623975 5874 catalog_manager.cc:1259] Loaded cluster ID: 592bd9c1dd1f40e2814f76ee20f87d15
I20260807 15:09:10.624007 5874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:10.624269 5861 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:09:10.625149 5874 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:10.625340 5874 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Loaded TSK: 0
I20260807 15:09:10.625566 5874 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:09:10.728859 5865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:10.729190 5865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:10.729245 5865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:10.730913 5865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:10.730971 5865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:10.731011 5865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:09:10.733225 5865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.129.20:37959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:36357
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=35089
--tserver_master_addrs=127.18.129.62:46587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:10.733603 5865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:10.733909 5865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:10.741056 5917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.741465 5916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.741117 5919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:10.743017 5865 server_base.cc:1034] running on GCE node
I20260807 15:09:10.743250 5865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:10.743520 5865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:10.744685 5865 hybrid_clock.cc:648] HybridClock initialized: now 1786115350744652 us; error 48 us; skew 500 ppm
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:10.748102 5865 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:09:10.748639 5865 webserver.cc:460] Webserver started at http://127.18.129.1:35089/ using document root <none> and password file <none>
I20260807 15:09:10.748925 5865 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:10.749012 5865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:10.777719 5865 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260807 15:09:10.779042 5936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.779315 5865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:10.779424 5865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "e455a293eb3849b7b483d28992e6e99f"
format_stamp: "Formatted at 2026-08-07 15:09:09 on dist-test-slave-5bn2"
server_key: "aa961580906f0915d3b6269c81439161"
server_key_iv: "bb647de65c08b42d0d48b2f77d8ac628"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.779920 5865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:10.802088 5865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:10.804436 5865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:10.804585 5865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:10.804826 5865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:10.805172 5865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:10.805217 5865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.805245 5865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:10.805263 5865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.811604 5865 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:36357
I20260807 15:09:10.812121 5865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:10.812803 6099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:36357 every 8 connection(s)
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:10.820802 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 5865
I20260807 15:09:10.821105 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:37959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:34019
--local_ip_for_outbound_sockets=127.18.129.2
--tserver_master_addrs=127.18.129.62:46587
--webserver_port=41127
--webserver_interface=127.18.129.2
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.129.20:37959/kms with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:10.825044 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:10.813603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46587 (local address 127.18.129.1:33073)
0807 15:09:10.814937 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:10.814974 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:10.815467 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:10.815939 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:10.815944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:10.816158 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:10.816356 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:10.816361 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.817447 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.817448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:10.818256 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:10.818259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.818887 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.818888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:10.819014 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:10.819495 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:10.819504 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:10.820772 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:10.823237 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:10.823244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:10.823246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:10.823542 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:10.823829 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:10.823830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:10.823830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:10.823958 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:10.824403 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:10.824410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:10.824558 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 15:09:10.824655 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 15:09:10.824816 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1289,"thread_start_us":91,"threads_started":1}
I20260807 15:09:10.825721 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:10.813676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:33073 (local address 127.18.129.62:46587)
0807 15:09:10.813882 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:10.813890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:10.815095 (+ 1205us) server_negotiation.cc:407] Connection header received
0807 15:09:10.815719 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:10.815724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:10.815769 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:10.815880 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:10.816473 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.817347 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.818340 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:10.818752 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:10.818875 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:10.820950 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:10.820955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:10.820956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:10.820968 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:10.823147 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:10.823624 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:10.823628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:10.823629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:10.823754 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:10.824030 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:10.824030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:10.824031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:10.824323 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:10.825330 (+ 1007us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:10.825404 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 15:09:10.825523 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":75,"threads_started":1}
I20260807 15:09:10.826529 6100 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46587
I20260807 15:09:10.826656 6100 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:10.826890 6100 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
W20260807 15:09:10.827840 6100 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:46587 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:09:10.827930 6100 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:10.828006 6100 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:10.828358 6100 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:10.828446 6100 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
W20260807 15:09:10.828745 6100 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:09:10.945101 6105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:10.945396 6105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:10.945434 6105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:10.947137 6105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:10.947198 6105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:10.947247 6105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:09:10.949447 6105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.129.20:37959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:34019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=41127
--tserver_master_addrs=127.18.129.62:46587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:10.949709 6105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:10.949991 6105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:10.955924 6114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.955932 6112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:10.956291 6117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:10.956452 6105 server_base.cc:1034] running on GCE node
I20260807 15:09:10.956666 6105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:10.956943 6105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:10.958134 6105 hybrid_clock.cc:648] HybridClock initialized: now 1786115350958107 us; error 40 us; skew 500 ppm
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:10.961295 6105 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:09:10.961732 6105 webserver.cc:460] Webserver started at http://127.18.129.2:41127/ using document root <none> and password file <none>
I20260807 15:09:10.962004 6105 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:10.962080 6105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:09:10 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:10.976307 6105 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:10.977466 6128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.977682 6105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:09:10.977782 6105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "2e8ed9a256094603ac3c1473058437ed"
format_stamp: "Formatted at 2026-08-07 15:09:10 on dist-test-slave-5bn2"
server_key: "d3ce3ea922af6a26f71baf2cc5156c30"
server_key_iv: "f52239c4ab549adde76ed6ecd34f811a"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:10.978160 6105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:10.993568 6105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:10.995657 6105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:10.995819 6105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:10.996076 6105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:10.996443 6105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:10.996487 6105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:10.996517 6105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:10.996551 6105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:11.003784 6105 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:34019
I20260807 15:09:11.003917 6322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:34019 every 8 connection(s)
I20260807 15:09:11.004245 6105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:09:11 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115350, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:11.010252 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 6105
I20260807 15:09:11.010548 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.129.20:37959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:44973
--local_ip_for_outbound_sockets=127.18.129.3
--tserver_master_addrs=127.18.129.62:46587
--webserver_port=37083
--webserver_interface=127.18.129.3
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.129.20:37959/kms with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:11.013763 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:11.005014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:55181 (local address 127.18.129.62:46587)
0807 15:09:11.005080 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:11.005082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:11.005332 (+ 250us) server_negotiation.cc:407] Connection header received
0807 15:09:11.005839 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:11.005841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:11.005857 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:11.005908 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:11.006462 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.007025 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.008021 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.008232 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.008288 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:11.010407 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:11.010409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:11.010409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:11.010417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:11.012170 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:11.012707 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:11.012709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:11.012709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:11.012781 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:11.013104 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:11.013105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:11.013105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:11.013240 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:11.013294 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:11.013627 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 15:09:11.013696 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:09:11.014020 6326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:11.005032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46587 (local address 127.18.129.2:55181)
0807 15:09:11.005204 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:11.005240 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:11.005734 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:11.005981 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:11.005984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:11.006164 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:11.006364 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:11.006368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.007126 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.007127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:11.007936 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:11.007938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.008322 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.008323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:11.008431 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:11.008926 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:11.008934 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:11.010232 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:11.012273 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:11.012279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:11.012281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:11.012577 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:11.012858 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:11.012858 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:11.012859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:11.013012 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:11.013302 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:11.013308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:11.013483 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 15:09:11.013610 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 15:09:11.013779 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":110,"threads_started":1}
I20260807 15:09:11.014508 6323 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46587
I20260807 15:09:11.014691 6323 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:11.014992 6323 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
W20260807 15:09:11.015712 6323 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:46587 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:09:11.015841 6323 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:11.015951 6323 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:11.016306 6323 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:11.016426 6323 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
W20260807 15:09:11.016758 6323 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:09:11.113080 6329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:11.113282 6329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:11.113312 6329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:11.114442 6329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:11.114488 6329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:11.114521 6329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:09:11.116019 6329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.129.20:37959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:44973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=37083
--tserver_master_addrs=127.18.129.62:46587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:11.116254 6329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:11.116542 6329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:11.130577 6342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:11.130878 6338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:11.131022 6337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:11.131139 6329 server_base.cc:1034] running on GCE node
I20260807 15:09:11.131404 6329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:11.131768 6329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:11.133010 6329 hybrid_clock.cc:648] HybridClock initialized: now 1786115351132982 us; error 44 us; skew 500 ppm
Aug 07 15:09:11 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115351, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:11.136461 6329 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:09:11.137036 6329 webserver.cc:460] Webserver started at http://127.18.129.3:37083/ using document root <none> and password file <none>
I20260807 15:09:11.137346 6329 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:11.137442 6329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:09:11 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115351, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:11.151073 6329 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:11.152060 6352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:11.152221 6329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:11.152303 6329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d"
format_stamp: "Formatted at 2026-08-07 15:09:10 on dist-test-slave-5bn2"
server_key: "92bf7c080a2232faa5e1d5d6dad1ce25"
server_key_iv: "c40eddcb752efd0227c16c7d8cec4340"
server_key_version: "kuduclusterkey@0"
I20260807 15:09:11.152742 6329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:11.174786 6329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:11.177213 6329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:11.177433 6329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:11.177767 6329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:11.178237 6329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:11.178337 6329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:11.178422 6329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:11.178459 6329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:11.186219 6329 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:44973
I20260807 15:09:11.186738 6329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:11.187477 6531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:44973 every 8 connection(s)
Aug 07 15:09:11 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115351, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:11.192829 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 6329
I20260807 15:09:11.196123 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:11.187904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:34843 (local address 127.18.129.62:46587)
0807 15:09:11.187953 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:11.187955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:11.187965 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:11.188582 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:11.188584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:11.188597 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:11.188649 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:11.189228 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.189728 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.190771 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.191071 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.191132 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:11.193358 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:11.193359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:11.193359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:11.193367 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:11.194715 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:11.195127 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:11.195128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:11.195128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:11.195186 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:11.195407 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:11.195408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:11.195408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:11.195543 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:11.195605 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:11.195945 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 15:09:11.196028 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:09:11.196525 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:11.187607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46587 (local address 127.18.129.3:34843)
0807 15:09:11.187772 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:11.187811 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:11.188472 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:11.188719 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:11.188723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:11.188932 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:11.189136 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:11.189141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.189812 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.189813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:11.190690 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:11.190693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.191160 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.191161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:11.191275 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:11.191790 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:11.191799 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:11.193280 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:11.194777 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:11.194783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:11.194785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:11.195060 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:11.195240 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:11.195241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:11.195242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:11.195352 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:11.195632 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:11.195639 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:11.195801 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 15:09:11.196220 (+ 419us) client_negotiation.cc:240] Negotiation successful
0807 15:09:11.196364 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260807 15:09:11.196960 6532 heartbeater.cc:347] Connected to a master server at 127.18.129.62:46587
I20260807 15:09:11.197050 6532 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:11.197247 6532 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
W20260807 15:09:11.197875 6532 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:46587 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:09:11.197947 6532 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:11.198004 6532 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:11.198279 6532 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:11.198359 6532 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
W20260807 15:09:11.198805 6532 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:09:11.204144 6539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:11.194212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.18.129.62:46587)
0807 15:09:11.194361 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:11.194363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:11.194531 (+ 168us) server_negotiation.cc:407] Connection header received
0807 15:09:11.194586 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:11.194587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:11.194600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:11.194677 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:11.195431 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.196141 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.197734 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:11.198023 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:11.198095 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:11.199498 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:11.199499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:11.199499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:11.199506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:11.201136 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:11.203262 (+ 2126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:11.203263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:11.203263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:11.203327 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:11.203532 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:11.203533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:11.203534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:11.203663 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:11.203851 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:11.203936 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 15:09:11.204026 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
Aug 07 15:09:11 dist-test-slave-5bn2 krb5kdc[4306](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115351, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:11.828963 6100 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:11.829144 6100 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:11.829523 5799 ts_manager.cc:194] Registered new tserver with Master: e455a293eb3849b7b483d28992e6e99f (127.18.129.1:36357)
I20260807 15:09:11.830217 5799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:33073
I20260807 15:09:12.016983 6323 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:12.017166 6323 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:12.017547 5799 ts_manager.cc:194] Registered new tserver with Master: 2e8ed9a256094603ac3c1473058437ed (127.18.129.2:34019)
I20260807 15:09:12.018078 5799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:55181
Aug 07 15:09:12 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115351, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:12.199007 6532 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:12.199177 6532 heartbeater.cc:511] Master 127.18.129.62:46587 requested a full tablet report, sending...
I20260807 15:09:12.199538 5799 ts_manager.cc:194] Registered new tserver with Master: 8fc9dc6315f44fcdbb8745d3e17bca1d (127.18.129.3:44973)
I20260807 15:09:12.200083 5799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:34843
I20260807 15:09:12.204560 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:12.213647 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.205509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.18.129.62:46587)
0807 15:09:12.205637 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:12.205639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:12.205649 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:12.205706 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:12.205707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:12.205720 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:12.205808 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:12.206462 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.207263 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.208254 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.208574 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.208658 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.210021 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:12.210023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:12.210023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:12.210032 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:12.211965 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:12.212420 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:12.212421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:12.212422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:12.212497 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:12.212737 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:12.212737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:12.212738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:12.212879 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:12.212987 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:12.213441 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 15:09:12.213528 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260807 15:09:12.216183 5799 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:12.218891 5799 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:12.227717 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.222318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36357 (local address 127.0.0.1:32886)
0807 15:09:12.222513 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:12.222557 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:12.222636 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:12.223452 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:12.223456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:12.223459 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:12.223535 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.223537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.224894 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.224895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.226309 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.226313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.227403 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.227404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.227494 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.227512 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:09:12.227570 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:09:12.227627 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":111,"threads_started":1}
I20260807 15:09:12.228319 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.223082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32886 (local address 127.18.129.1:36357)
0807 15:09:12.223255 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:12.223260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:12.223270 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:12.223305 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:12.223307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:12.223323 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:12.223422 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:12.223646 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.224777 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.226462 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.227256 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.228085 (+ 829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.228162 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:12.228193 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 15:09:12.228234 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":74,"threads_started":1}
I20260807 15:09:12.229225 5998 tablet_service.cc:1467] Processing CreateTablet for tablet 4fbb61e609b94a13acad0ae7e9caba8e (DEFAULT_TABLE table=test-table [id=4dfac4316b3b4113bdc8fed82b7266ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:12.229653 5998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fbb61e609b94a13acad0ae7e9caba8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:12.232959 6576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.222623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:34019 (local address 127.0.0.1:34744)
0807 15:09:12.222773 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:12.222782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:12.222817 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:12.227177 (+ 4360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:12.227178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:12.227180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:12.227237 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.227239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.228895 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.228896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.229991 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.229994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.232749 (+ 2755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.232750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.232821 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.232825 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:12.232851 (+ 26us) client_negotiation.cc:240] Negotiation successful
0807 15:09:12.232890 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":110,"threads_started":1}
I20260807 15:09:12.233314 6581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.223238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.18.129.2:34019)
0807 15:09:12.226991 (+ 3753us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:12.226996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:12.227007 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:09:12.227053 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:12.227055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:12.227070 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:12.227306 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:12.227388 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.228791 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.230119 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.232652 (+ 2533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.233112 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.233181 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:12.233205 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 15:09:12.233243 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3714,"thread_start_us":55,"threads_started":1}
I20260807 15:09:12.234089 6207 tablet_service.cc:1467] Processing CreateTablet for tablet 4fbb61e609b94a13acad0ae7e9caba8e (DEFAULT_TABLE table=test-table [id=4dfac4316b3b4113bdc8fed82b7266ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:12.234405 6207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fbb61e609b94a13acad0ae7e9caba8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:12.235888 6577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.222838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:44973 (local address 127.0.0.1:41400)
0807 15:09:12.222989 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:12.222998 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:12.223058 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:12.231093 (+ 8035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:12.231094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:12.231096 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:12.231136 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.231137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.233447 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.233448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.234534 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.234543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.235702 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.235703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.235759 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.235764 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:09:12.235788 (+ 24us) client_negotiation.cc:240] Negotiation successful
0807 15:09:12.235824 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260807 15:09:12.236395 6582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.223374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.18.129.3:44973)
0807 15:09:12.230920 (+ 7546us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:12.230925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:12.230936 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:09:12.230968 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:12.230969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:12.230986 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:12.231205 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:12.231279 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.232253 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.234703 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.235608 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.236182 (+ 574us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.236250 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:12.236277 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 15:09:12.236314 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7492,"thread_start_us":58,"threads_started":1}
I20260807 15:09:12.238013 6585 tablet_bootstrap.cc:492] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Bootstrap starting.
I20260807 15:09:12.238714 6585 tablet_bootstrap.cc:654] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:12.238710 6410 tablet_service.cc:1467] Processing CreateTablet for tablet 4fbb61e609b94a13acad0ae7e9caba8e (DEFAULT_TABLE table=test-table [id=4dfac4316b3b4113bdc8fed82b7266ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:12.239115 6410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fbb61e609b94a13acad0ae7e9caba8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:12.239254 6585 log.cc:824] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:12.240029 6585 tablet_bootstrap.cc:492] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: No bootstrap required, opened a new log
I20260807 15:09:12.240087 6585 ts_tablet_manager.cc:1397] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:12.242511 6585 raft_consensus.cc:348] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.242669 6585 raft_consensus.cc:374] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:12.242700 6585 raft_consensus.cc:729] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8ed9a256094603ac3c1473058437ed, State: Initialized, Role: FOLLOWER
I20260807 15:09:12.242882 6585 consensus_queue.cc:260] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.242993 6584 tablet_bootstrap.cc:492] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f: Bootstrap starting.
I20260807 15:09:12.243155 6585 ts_tablet_manager.cc:1428] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:12.243269 6323 heartbeater.cc:503] Master 127.18.129.62:46587 was elected leader, sending a full tablet report...
I20260807 15:09:12.243638 6584 tablet_bootstrap.cc:654] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:12.243638 6587 tablet_bootstrap.cc:492] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: Bootstrap starting.
I20260807 15:09:12.243942 6584 log.cc:824] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:12.244251 6587 tablet_bootstrap.cc:654] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:12.244529 6587 log.cc:824] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:12.245337 6587 tablet_bootstrap.cc:492] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: No bootstrap required, opened a new log
I20260807 15:09:12.245406 6587 ts_tablet_manager.cc:1397] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:12.247627 6587 raft_consensus.cc:348] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.247792 6587 raft_consensus.cc:374] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:12.247824 6587 raft_consensus.cc:729] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fc9dc6315f44fcdbb8745d3e17bca1d, State: Initialized, Role: FOLLOWER
I20260807 15:09:12.247968 6587 consensus_queue.cc:260] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.248231 6587 ts_tablet_manager.cc:1428] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:09:12.248348 6532 heartbeater.cc:503] Master 127.18.129.62:46587 was elected leader, sending a full tablet report...
I20260807 15:09:12.249313 6584 tablet_bootstrap.cc:492] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f: No bootstrap required, opened a new log
I20260807 15:09:12.249392 6584 ts_tablet_manager.cc:1397] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 15:09:12.251462 6584 raft_consensus.cc:348] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.251639 6584 raft_consensus.cc:374] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:12.251669 6584 raft_consensus.cc:729] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e455a293eb3849b7b483d28992e6e99f, State: Initialized, Role: FOLLOWER
I20260807 15:09:12.251803 6584 consensus_queue.cc:260] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.252045 6584 ts_tablet_manager.cc:1428] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:12.252156 6100 heartbeater.cc:503] Master 127.18.129.62:46587 was elected leader, sending a full tablet report...
W20260807 15:09:12.255385 6325 tablet.cc:2369] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:09:12.313992 6101 tablet.cc:2369] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:12.316257 6593 raft_consensus.cc:482] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:12.316390 6593 raft_consensus.cc:504] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.316789 6593 leader_election.cc:288] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8ed9a256094603ac3c1473058437ed (127.18.129.2:34019), 8fc9dc6315f44fcdbb8745d3e17bca1d (127.18.129.3:44973)
I20260807 15:09:12.321234 6581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.317092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:35351 (local address 127.18.129.2:34019)
0807 15:09:12.317168 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:12.317170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:12.317500 (+ 330us) server_negotiation.cc:407] Connection header received
0807 15:09:12.317543 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:12.317544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:12.317556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:12.317623 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:12.318298 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.318985 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.320249 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.320817 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.320867 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.320877 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:12.321117 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 15:09:12.321160 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:09:12.321496 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.316985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:34019 (local address 127.18.129.1:35351)
0807 15:09:12.317393 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:12.317404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:12.317483 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:12.317687 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:12.317688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:12.317691 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:12.317778 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.317780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.319068 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.319069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.320143 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.320146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.320966 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.320967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.321039 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.321042 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:09:12.321356 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 15:09:12.321399 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":109,"threads_started":1}
I20260807 15:09:12.321813 6216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fbb61e609b94a13acad0ae7e9caba8e" candidate_uuid: "e455a293eb3849b7b483d28992e6e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8ed9a256094603ac3c1473058437ed" is_pre_election: true
I20260807 15:09:12.322010 6216 raft_consensus.cc:2454] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e455a293eb3849b7b483d28992e6e99f in term 0.
I20260807 15:09:12.322259 5938 leader_election.cc:302] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e8ed9a256094603ac3c1473058437ed, e455a293eb3849b7b483d28992e6e99f; no voters:
I20260807 15:09:12.322441 6593 raft_consensus.cc:2781] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:12.322504 6593 raft_consensus.cc:482] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:12.322533 6593 raft_consensus.cc:3037] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:12.323518 6593 raft_consensus.cc:504] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.323652 6593 leader_election.cc:288] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [CANDIDATE]: Term 1 election: Requested vote from peers 2e8ed9a256094603ac3c1473058437ed (127.18.129.2:34019), 8fc9dc6315f44fcdbb8745d3e17bca1d (127.18.129.3:44973)
I20260807 15:09:12.323863 6216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fbb61e609b94a13acad0ae7e9caba8e" candidate_uuid: "e455a293eb3849b7b483d28992e6e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8ed9a256094603ac3c1473058437ed"
I20260807 15:09:12.323957 6216 raft_consensus.cc:3037] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:12.324774 6582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.317289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:35309 (local address 127.18.129.3:44973)
0807 15:09:12.317327 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:12.317329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:12.317620 (+ 291us) server_negotiation.cc:407] Connection header received
0807 15:09:12.317707 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:12.317708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:12.317717 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:12.317744 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:12.317992 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.318720 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.323207 (+ 4487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.323737 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.323788 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.323798 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:12.324655 (+ 857us) server_negotiation.cc:299] Negotiation successful
0807 15:09:12.324706 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:09:12.324826 6216 raft_consensus.cc:2454] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e455a293eb3849b7b483d28992e6e99f in term 1.
I20260807 15:09:12.324990 5938 leader_election.cc:302] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e8ed9a256094603ac3c1473058437ed, e455a293eb3849b7b483d28992e6e99f; no voters:
I20260807 15:09:12.325019 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.317210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:44973 (local address 127.18.129.1:35309)
0807 15:09:12.317572 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:12.317579 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:12.317639 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:12.317807 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:12.317808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:12.317810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:12.317878 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.317879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.318808 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.318809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.323105 (+ 4296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:12.323108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.324351 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.324352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:12.324422 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.324425 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:09:12.324895 (+ 470us) client_negotiation.cc:240] Negotiation successful
0807 15:09:12.324941 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":72,"threads_started":1}
I20260807 15:09:12.325142 6593 raft_consensus.cc:2781] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:12.325290 6593 raft_consensus.cc:686] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [term 1 LEADER]: Becoming Leader. State: Replica: e455a293eb3849b7b483d28992e6e99f, State: Running, Role: LEADER
I20260807 15:09:12.325309 6461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fbb61e609b94a13acad0ae7e9caba8e" candidate_uuid: "e455a293eb3849b7b483d28992e6e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" is_pre_election: true
I20260807 15:09:12.325389 6593 consensus_queue.cc:237] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } }
I20260807 15:09:12.325472 6461 raft_consensus.cc:2454] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e455a293eb3849b7b483d28992e6e99f in term 0.
I20260807 15:09:12.325611 6460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fbb61e609b94a13acad0ae7e9caba8e" candidate_uuid: "e455a293eb3849b7b483d28992e6e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d"
I20260807 15:09:12.325680 6460 raft_consensus.cc:3037] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:12.326197 5799 catalog_manager.cc:5495] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f reported cstate change: term changed from 0 to 1, leader changed from <none> to e455a293eb3849b7b483d28992e6e99f (127.18.129.1). New cstate: current_term: 1 leader_uuid: "e455a293eb3849b7b483d28992e6e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e455a293eb3849b7b483d28992e6e99f" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 36357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 } health_report { overall_health: UNKNOWN } } }
I20260807 15:09:12.326496 6460 raft_consensus.cc:2454] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e455a293eb3849b7b483d28992e6e99f in term 1.
Aug 07 15:09:12 dist-test-slave-5bn2 krb5kdc[4306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115351, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for HTTP/127.18.129.20@KRBTEST.COM
I20260807 15:09:12.352715 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.349174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.18.129.1:36357)
0807 15:09:12.349224 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:12.349226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:12.349235 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:09:12.349280 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:12.349281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:12.349304 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:12.349384 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:12.349571 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.350349 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.351528 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:12.351822 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:12.351892 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:12.352132 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:12.352276 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:12.352366 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:12.352605 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 15:09:12.352627 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:09:12.359771 6460 raft_consensus.cc:1261] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 1 FOLLOWER]: Refusing update from remote peer e455a293eb3849b7b483d28992e6e99f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:12.360219 6593 consensus_queue.cc:1035] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fc9dc6315f44fcdbb8745d3e17bca1d" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:12.360553 6216 raft_consensus.cc:1261] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 1 FOLLOWER]: Refusing update from remote peer e455a293eb3849b7b483d28992e6e99f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:12.361006 6593 consensus_queue.cc:1035] T 4fbb61e609b94a13acad0ae7e9caba8e P e455a293eb3849b7b483d28992e6e99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8ed9a256094603ac3c1473058437ed" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 34019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:12.365278 6607 mvcc.cc:205] Tried to move back new op lower bound from 7315928483256225792 to 7315928483125370880. Current Snapshot: MvccSnapshot[applied={T|T < 7315928483256225792}]
I20260807 15:09:12.371547 5799 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:12.371660 5799 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:12.372427 5799 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83c7089f938340a3b284d5acb8b83428: Sending DeleteTablet for 3 replicas of tablet 4fbb61e609b94a13acad0ae7e9caba8e
I20260807 15:09:12.372834 5998 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fbb61e609b94a13acad0ae7e9caba8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:12 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:32886
I20260807 15:09:12.372850 6613 mvcc.cc:205] Tried to move back new op lower bound from 7315928483256225792 to 7315928483125370880. Current Snapshot: MvccSnapshot[applied={T|T < 7315928483256225792}]
I20260807 15:09:12.373109 6207 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fbb61e609b94a13acad0ae7e9caba8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:12 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:34744
I20260807 15:09:12.373560 6410 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fbb61e609b94a13acad0ae7e9caba8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:12 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41400
I20260807 15:09:12.373802 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 5865
I20260807 15:09:12.387333 6622 tablet_replica.cc:330] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: stopping tablet replica
I20260807 15:09:12.387490 6622 raft_consensus.cc:2229] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:12.387624 6622 raft_consensus.cc:2258] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:12.387892 6623 tablet_replica.cc:330] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: stopping tablet replica
I20260807 15:09:12.388019 6623 raft_consensus.cc:2229] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:12.388134 6623 raft_consensus.cc:2258] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:12.388511 6622 ts_tablet_manager.cc:1905] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:12.390390 6622 ts_tablet_manager.cc:1918] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:12.390488 6622 log.cc:1196] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/4fbb61e609b94a13acad0ae7e9caba8e
I20260807 15:09:12.390822 6622 ts_tablet_manager.cc:1939] T 4fbb61e609b94a13acad0ae7e9caba8e P 2e8ed9a256094603ac3c1473058437ed: Deleting consensus metadata
I20260807 15:09:12.391273 5778 catalog_manager.cc:4841] TS 2e8ed9a256094603ac3c1473058437ed (127.18.129.2:34019): tablet 4fbb61e609b94a13acad0ae7e9caba8e (table test-table [id=4dfac4316b3b4113bdc8fed82b7266ba]) successfully deleted
I20260807 15:09:12.391644 6623 ts_tablet_manager.cc:1905] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:12.393206 6623 ts_tablet_manager.cc:1918] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:12.393287 6623 log.cc:1196] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/4fbb61e609b94a13acad0ae7e9caba8e
I20260807 15:09:12.393587 6623 ts_tablet_manager.cc:1939] T 4fbb61e609b94a13acad0ae7e9caba8e P 8fc9dc6315f44fcdbb8745d3e17bca1d: Deleting consensus metadata
I20260807 15:09:12.394006 5778 catalog_manager.cc:4841] TS 8fc9dc6315f44fcdbb8745d3e17bca1d (127.18.129.3:44973): tablet 4fbb61e609b94a13acad0ae7e9caba8e (table test-table [id=4dfac4316b3b4113bdc8fed82b7266ba]) successfully deleted
W20260807 15:09:12.397257 5778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:36357 (error 108)
I20260807 15:09:12.397619 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 6105
I20260807 15:09:12.420634 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 6329
I20260807 15:09:12.427770 6577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:12.420816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:36357 (local address 127.0.0.1:32912)
0807 15:09:12.427634 (+ 6818us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:12.427716 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:36357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6784}
W20260807 15:09:12.442361 5778 catalog_manager.cc:4584] TS e455a293eb3849b7b483d28992e6e99f (127.18.129.1:36357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4fbb61e609b94a13acad0ae7e9caba8e: Network error: Client connection negotiation failed: client connection to 127.18.129.1:36357: connect: Connection refused (error 111)
I20260807 15:09:12.449896 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 5756
I20260807 15:09:12.464788 18948 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:09:12.808328 18948 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:09:12.808429 18948 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:09:13.152842 18948 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:09:13.153 UTC [4354] LOG: received smart shutdown request
2026-08-07 15:09:13.157 UTC [4354] LOG: background worker "logical replication launcher" (PID 4366) exited with exit code 1
2026-08-07 15:09:13.157 UTC [4360] LOG: shutting down
2026-08-07 15:09:13.310 UTC [4354] LOG: database system is shut down
2026-08-07T15:09:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49437 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:13 dist-test-slave-5bn2 krb5kdc[6642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:13 dist-test-slave-5bn2 krb5kdc[6642](info): set up 2 sockets
Aug 07 15:09:13 dist-test-slave-5bn2 krb5kdc[6642](info): commencing operation
krb5kdc: starting...
W20260807 15:09:14.530766 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:09:14 dist-test-slave-5bn2 krb5kdc[6642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:09:14.647603 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43757
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:43757,127.18.129.61:38795,127.18.129.60:46623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:14.729308 6663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:14.729518 6663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:14.729547 6663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:14.730701 6663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:14.730747 6663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:14.730759 6663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:14.730772 6663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:14.730785 6663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:14.732369 6663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:43757,127.18.129.61:38795,127.18.129.60:46623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:14.732621 6663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:14.732839 6663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:14.737670 6671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:14.737701 6669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:14.737699 6668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:14.737771 6663 server_base.cc:1034] running on GCE node
I20260807 15:09:14.738147 6663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:14.738425 6663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:14.739568 6663 hybrid_clock.cc:648] HybridClock initialized: now 1786115354739554 us; error 30 us; skew 500 ppm
Aug 07 15:09:14 dist-test-slave-5bn2 krb5kdc[6642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:14.744740 6663 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:14.745210 6663 webserver.cc:460] Webserver started at http://127.18.129.62:40203/ using document root <none> and password file <none>
I20260807 15:09:14.745440 6663 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:14.745491 6663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:14.745630 6663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:14.746703 6663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "d16f87bf36394443884d6692ab75f9cc"
format_stamp: "Formatted at 2026-08-07 15:09:14 on dist-test-slave-5bn2"
server_key: "4146cda9ef0ceb9dafd7865e5d70a3b9"
server_key_iv: "025c3e8404bf4b002290d08aa51d7b96"
server_key_version: "encryptionkey@0"
I20260807 15:09:14.747088 6663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "d16f87bf36394443884d6692ab75f9cc"
format_stamp: "Formatted at 2026-08-07 15:09:14 on dist-test-slave-5bn2"
server_key: "4146cda9ef0ceb9dafd7865e5d70a3b9"
server_key_iv: "025c3e8404bf4b002290d08aa51d7b96"
server_key_version: "encryptionkey@0"
I20260807 15:09:14.748414 6663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:14.749130 6680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:14.749289 6663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:09:14.749367 6663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "d16f87bf36394443884d6692ab75f9cc"
format_stamp: "Formatted at 2026-08-07 15:09:14 on dist-test-slave-5bn2"
server_key: "4146cda9ef0ceb9dafd7865e5d70a3b9"
server_key_iv: "025c3e8404bf4b002290d08aa51d7b96"
server_key_version: "encryptionkey@0"
I20260807 15:09:14.749430 6663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:14.770660 6663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:14.772701 6663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:14.772850 6663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:14.776870 6663 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:43757
I20260807 15:09:14.776904 6761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:43757 every 8 connection(s)
I20260807 15:09:14.777243 6663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:14.778124 6762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:14.779767 6762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:09:14.782640 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 6663
I20260807 15:09:14.782739 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:09:14.782938 18948 external_mini_cluster.cc:1428] Setting key 6b6ce783c526c1b785fdac74775a8993
Aug 07 15:09:14 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:14.788830 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.780551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.18.129.62:43757)
0807 15:09:14.780672 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:14.780677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:14.780706 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:09:14.781137 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:14.781139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:14.781155 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:14.781229 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:14.781733 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.782545 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.783498 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.783809 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.783904 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:14.785844 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:14.785851 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:14.785852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:14.785859 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:14.787287 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:14.787736 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:14.787738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:14.787738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:14.787819 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:14.788063 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:14.788064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:14.788064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:14.788257 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:14.788315 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:14.788530 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 15:09:14.788644 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":90,"thread_start_us":42,"threads_started":1}
I20260807 15:09:14.788851 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.780389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43757 (local address 127.0.0.1:44384)
0807 15:09:14.780544 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.780580 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:14.781016 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:14.781242 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:14.781243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:14.781493 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:14.781645 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:14.781649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.782637 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.782638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:14.783420 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:14.783423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.783882 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.783883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:14.783950 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:14.784435 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:14.784443 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:14.785765 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:14.787382 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:14.787391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:14.787393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:14.787663 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:14.787877 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:14.787878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:14.787878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:14.787999 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:14.788319 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:14.788326 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:14.788405 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 15:09:14.788507 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:09:14.788709 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":53,"thread_start_us":68,"threads_started":1}
I20260807 15:09:14.789594 6762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:09:14.789940 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.789802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40922)
0807 15:09:14.789852 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.789908 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 15:09:14.790089 6681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
I20260807 15:09:14.790515 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.790425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40926)
0807 15:09:14.790469 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.790496 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:09:14.790628 6762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:38795: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.129.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab.
Entry for principal kudu/127.18.129.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab.
I20260807 15:09:14.807890 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.61:38795
--webserver_interface=127.18.129.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab
--principal=kudu/127.18.129.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:43757,127.18.129.61:38795,127.18.129.60:46623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:14.839831 6762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } attempt: 1
I20260807 15:09:14.840284 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.840166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40938)
0807 15:09:14.840222 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.840254 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:09:14.840595 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.840508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40942)
0807 15:09:14.840551 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.840574 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 15:09:14.840682 6762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:38795: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
W20260807 15:09:14.891348 6773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:14.891561 6773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:14.891602 6773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:14.892742 6773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:14.892783 6773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:14.892802 6773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:14.892817 6773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:14.892833 6773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:14.894358 6773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:43757,127.18.129.61:38795,127.18.129.60:46623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab
--principal=kudu/127.18.129.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.61:38795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.129.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:14.894608 6773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:14.894807 6773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:14.899730 6780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:14.899880 6784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:14.899894 6781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:14.899730 6773 server_base.cc:1034] running on GCE node
I20260807 15:09:14.900228 6773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:14.900518 6773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:14.901670 6773 hybrid_clock.cc:648] HybridClock initialized: now 1786115354901656 us; error 38 us; skew 500 ppm
Aug 07 15:09:14 dist-test-slave-5bn2 krb5kdc[6642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:14.904181 6773 init.cc:377] Logged in from keytab as kudu/127.18.129.61@KRBTEST.COM (short username kudu)
I20260807 15:09:14.904618 6773 webserver.cc:460] Webserver started at http://127.18.129.61:43411/ using document root <none> and password file <none>
I20260807 15:09:14.904840 6773 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:14.904891 6773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:14.905015 6773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:14.906056 6773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/data/instance:
uuid: "5fb13c82184b4cbabcf6b14c3d83eabc"
format_stamp: "Formatted at 2026-08-07 15:09:14 on dist-test-slave-5bn2"
server_key: "5de33e46a258b70ee0275df6cfb50679"
server_key_iv: "0421d1cf8dcf006467a90042ca3058c0"
server_key_version: "encryptionkey@0"
I20260807 15:09:14.906391 6773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/wal/instance:
uuid: "5fb13c82184b4cbabcf6b14c3d83eabc"
format_stamp: "Formatted at 2026-08-07 15:09:14 on dist-test-slave-5bn2"
server_key: "5de33e46a258b70ee0275df6cfb50679"
server_key_iv: "0421d1cf8dcf006467a90042ca3058c0"
server_key_version: "encryptionkey@0"
I20260807 15:09:14.907685 6773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:09:14.908353 6797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:14.908501 6773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:14.908604 6773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/wal
uuid: "5fb13c82184b4cbabcf6b14c3d83eabc"
format_stamp: "Formatted at 2026-08-07 15:09:14 on dist-test-slave-5bn2"
server_key: "5de33e46a258b70ee0275df6cfb50679"
server_key_iv: "0421d1cf8dcf006467a90042ca3058c0"
server_key_version: "encryptionkey@0"
I20260807 15:09:14.908668 6773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:14.908820 6762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } attempt: 2
I20260807 15:09:14.909189 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.909087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40946)
0807 15:09:14.909128 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.909162 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 15:09:14.909498 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.909418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40950)
0807 15:09:14.909456 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.909479 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 15:09:14.909585 6762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:38795: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38795: connect: Connection refused (error 111)
I20260807 15:09:14.930238 6773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:14.932458 6773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:14.932610 6773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:14.936726 6773 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.61:38795
I20260807 15:09:14.936767 6876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.61:38795 every 8 connection(s)
I20260807 15:09:14.937103 6773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
I20260807 15:09:14.937948 6877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:14.939567 6877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:09:14.943176 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 6773
I20260807 15:09:14.943295 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-1/wal/instance
I20260807 15:09:14.943465 18948 external_mini_cluster.cc:1428] Setting key 77c9146c88729d24ca0d77dce59f2c53
Aug 07 15:09:14 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:14.948621 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.940251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.18.129.62:43757)
0807 15:09:14.940308 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:14.940309 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:14.940492 (+ 183us) server_negotiation.cc:407] Connection header received
0807 15:09:14.940964 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:14.940965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:14.940982 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:14.941034 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:14.941648 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.942134 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.943174 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.943429 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.943484 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:14.945826 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:14.945827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:14.945828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:14.945835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:14.947132 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:14.947592 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:14.947592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:14.947593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:14.947648 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:14.947934 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:14.947934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:14.947935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:14.948030 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:14.948085 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:14.948423 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 15:09:14.948513 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:09:14.948750 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.940227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43757 (local address 127.0.0.1:44390)
0807 15:09:14.940380 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.940416 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:14.940866 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:14.941117 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:14.941121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:14.941364 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:14.941558 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:14.941562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.942216 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.942217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:14.943088 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:14.943091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.943525 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.943526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:14.943680 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:14.944240 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:14.944250 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:14.945741 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:14.947207 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:14.947215 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:14.947218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:14.947519 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:14.947710 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:14.947711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:14.947712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:14.947870 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:14.948102 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:14.948110 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:14.948247 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 15:09:14.948381 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 15:09:14.948538 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260807 15:09:14.949270 6877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.129.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.60@KRBTEST.COM" created.
Aug 07 15:09:14 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:09:14.958446 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.949523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40962)
0807 15:09:14.949572 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.949585 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:14.949630 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:14.949840 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:14.949840 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:14.949980 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:14.950043 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:14.950045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.950895 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.950896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:14.951658 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:14.951661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.952121 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.952122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:14.952195 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:14.952527 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:14.952532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:14.955807 (+ 3275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:14.957422 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:14.957423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:14.957423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:14.957626 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:14.957809 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:14.957810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:14.957810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:14.957870 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:14.958135 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:14.958135 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:14.958178 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:09:14.958240 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 15:09:14.958340 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:09:14.958510 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.949577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.18.129.61:38795)
0807 15:09:14.949714 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:14.949717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:14.949731 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:09:14.949749 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:14.949750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:14.949760 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:14.949822 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:14.950138 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.950789 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.951734 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:14.952055 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:14.952106 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:14.955881 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:14.955890 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:14.955890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:14.955897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:14.957341 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:14.957701 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:14.957704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:14.957704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:14.957755 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:14.957925 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:14.957926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:14.957927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:14.958071 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:14.958143 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:14.958277 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:09:14.958391 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260807 15:09:14.959076 6877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:09:14.959409 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.959302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:50974)
0807 15:09:14.959343 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.959387 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 15:09:14.959519 6801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
I20260807 15:09:14.959872 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:14.959788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:50990)
0807 15:09:14.959823 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:14.959847 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 15:09:14.960003 6877 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:46623: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab.
Entry for principal kudu/127.18.129.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab.
I20260807 15:09:14.970129 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.60:46623
--webserver_interface=127.18.129.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab
--principal=kudu/127.18.129.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:43757,127.18.129.61:38795,127.18.129.60:46623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:15.000715 6762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } attempt: 3
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:09:15.008021 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.001120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40966)
0807 15:09:15.001182 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.001196 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:15.001252 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:15.001383 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:15.001383 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:15.001532 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:15.001598 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.001600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.002508 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.002509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.003023 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.003025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.003491 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.003491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.003547 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.004013 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:15.004017 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:15.005389 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:15.007100 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.007101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.007101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.007265 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.007469 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.007469 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.007469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.007519 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.007780 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:15.007781 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:15.007822 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 15:09:15.007865 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:09:15.007945 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:09:15.008080 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.001176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.18.129.61:38795)
0807 15:09:15.001246 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.001248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.001262 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:09:15.001307 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.001308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.001322 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.001373 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.001661 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.002439 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.003102 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.003427 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.003497 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.005477 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.005479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.005479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.005487 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.007019 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.007333 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.007334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.007334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.007407 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.007581 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.007582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.007582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.007725 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.007796 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.007893 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.007982 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:09:15.008492 6762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:09:15.008829 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.008741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:50996)
0807 15:09:15.008781 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.008808 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 15:09:15.009126 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.009061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51002)
0807 15:09:15.009088 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.009111 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260807 15:09:15.009207 6762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:46623: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
I20260807 15:09:15.009235 6877 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } attempt: 1
I20260807 15:09:15.009544 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.009443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51004)
0807 15:09:15.009483 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.009509 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 15:09:15.009814 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.009758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51016)
0807 15:09:15.009781 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.009800 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260807 15:09:15.009891 6877 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:46623: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
I20260807 15:09:15.040391 6762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } attempt: 1
I20260807 15:09:15.040877 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.040723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51018)
0807 15:09:15.040779 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.040812 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:09:15.041214 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.041136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51032)
0807 15:09:15.041167 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.041193 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260807 15:09:15.041297 6762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:46623: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
W20260807 15:09:15.053840 6884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:15.054054 6884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:15.054095 6884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:15.055248 6884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:15.055291 6884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:15.055310 6884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:15.055325 6884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:15.055341 6884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:15.056877 6884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:43757,127.18.129.61:38795,127.18.129.60:46623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab
--principal=kudu/127.18.129.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.60:46623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.129.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:15.057140 6884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:15.057349 6884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:15.062041 6892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:15.062317 6894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:15.062359 6896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:15.062522 6884 server_base.cc:1034] running on GCE node
I20260807 15:09:15.062805 6884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:15.063131 6884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:15.064275 6884 hybrid_clock.cc:648] HybridClock initialized: now 1786115355064253 us; error 33 us; skew 500 ppm
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:15.066660 6884 init.cc:377] Logged in from keytab as kudu/127.18.129.60@KRBTEST.COM (short username kudu)
I20260807 15:09:15.067116 6884 webserver.cc:460] Webserver started at http://127.18.129.60:43969/ using document root <none> and password file <none>
I20260807 15:09:15.067322 6884 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:15.067371 6884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:15.067497 6884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:15.068549 6884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/data/instance:
uuid: "fc0516071bba4a8097911f1924d56005"
format_stamp: "Formatted at 2026-08-07 15:09:15 on dist-test-slave-5bn2"
server_key: "f66b9b0a987db077f90cb0047ecc7d29"
server_key_iv: "a62cff8137a9a2f551931cb3a680ae91"
server_key_version: "encryptionkey@0"
I20260807 15:09:15.068892 6884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/wal/instance:
uuid: "fc0516071bba4a8097911f1924d56005"
format_stamp: "Formatted at 2026-08-07 15:09:15 on dist-test-slave-5bn2"
server_key: "f66b9b0a987db077f90cb0047ecc7d29"
server_key_iv: "a62cff8137a9a2f551931cb3a680ae91"
server_key_version: "encryptionkey@0"
I20260807 15:09:15.070122 6884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:09:15.070749 6904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:15.070921 6884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:15.071009 6884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/wal
uuid: "fc0516071bba4a8097911f1924d56005"
format_stamp: "Formatted at 2026-08-07 15:09:15 on dist-test-slave-5bn2"
server_key: "f66b9b0a987db077f90cb0047ecc7d29"
server_key_iv: "a62cff8137a9a2f551931cb3a680ae91"
server_key_version: "encryptionkey@0"
I20260807 15:09:15.071087 6884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:15.078037 6877 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } attempt: 2
I20260807 15:09:15.078445 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.078304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51048)
0807 15:09:15.078360 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.078406 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:09:15.078735 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.078672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51060)
0807 15:09:15.078702 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.078721 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 15:09:15.078814 6877 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:46623: Network error: Client connection negotiation failed: client connection to 127.18.129.60:46623: connect: Connection refused (error 111)
I20260807 15:09:15.083175 6884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:15.085297 6884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:15.085429 6884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:15.089313 6884 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.60:46623
I20260807 15:09:15.089354 6988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.60:46623 every 8 connection(s)
I20260807 15:09:15.089664 6884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
I20260807 15:09:15.090547 6989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:15.092116 6989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:09:15.095028 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 6884
I20260807 15:09:15.095161 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115197351496-18948-0/minicluster-data/master-2/wal/instance
I20260807 15:09:15.095355 18948 external_mini_cluster.cc:1428] Setting key dc41b120b2579a5dd3269a2e54e65703
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:09:15.101297 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.092775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44394 (local address 127.18.129.62:43757)
0807 15:09:15.092831 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.092833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.093018 (+ 185us) server_negotiation.cc:407] Connection header received
0807 15:09:15.093481 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.093482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.093495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.093548 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.094155 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.094615 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.095950 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.096233 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.096292 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.098386 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.098388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.098388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.098395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.099857 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.100309 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.100310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.100310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.100362 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.100651 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.100652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.100653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.100762 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.100816 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.101137 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.101217 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:09:15.101436 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.092732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43757 (local address 127.0.0.1:44394)
0807 15:09:15.092907 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.092943 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:15.093387 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:15.093631 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:15.093635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:15.093873 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:15.094063 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.094068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.094692 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.094693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.095852 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.095855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.096303 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.096304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.096426 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.096902 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:15.096910 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:15.098314 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:15.099934 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.099939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.099941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.100236 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.100419 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.100420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.100420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.100551 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.100829 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:15.100836 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:15.100997 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 15:09:15.101086 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:09:15.101205 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260807 15:09:15.102191 6989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:09:15.104104 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.095725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.18.129.60:46623)
0807 15:09:15.095871 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.095875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.095899 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:09:15.095954 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.095956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.095995 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.096065 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.096720 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.097430 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.098241 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.098526 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.098586 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.100987 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.100994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.100994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.101000 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.103127 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.103422 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.103425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.103425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.103484 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.103651 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.103652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.103652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.103778 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.103900 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.103974 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.104038 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:09:15.108651 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.102441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38795 (local address 127.0.0.1:40970)
0807 15:09:15.102488 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.102499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:15.102537 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:15.102681 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:15.102682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:15.102790 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:15.102819 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.102820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.103600 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.103602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.104247 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.104249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.104628 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.104630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.104693 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.105032 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:15.105037 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:15.106301 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:15.107726 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.107727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.107727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.107929 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.108140 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.108141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.108141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.108192 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.108430 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:15.108430 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:15.108461 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 15:09:15.108503 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:09:15.108592 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 15:09:15.108707 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.102508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.18.129.61:38795)
0807 15:09:15.102572 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.102574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.102583 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:09:15.102609 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.102610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.102621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.102679 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.102906 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.103501 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.104310 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.104541 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.104606 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.106372 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.106373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.106374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.106380 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.107651 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.107996 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.107997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.107997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.108067 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.108242 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.108243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.108244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.108375 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.108441 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.108540 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.108613 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:09:15.109026 6989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:15.112398 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.104968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.18.129.62:43757)
0807 15:09:15.105007 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.105009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.105060 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:09:15.105146 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.105147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.105154 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.105192 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.105755 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.106428 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.107326 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.107504 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.107555 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.109818 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.109820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.109820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.109827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.111263 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.111615 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.111616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.111616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.111681 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.111866 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.111867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.111867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.111996 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.112191 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.112234 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.112309 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:09:15.115553 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.109262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51090)
0807 15:09:15.109289 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.109298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:15.109332 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:15.109429 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:15.109429 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:15.109557 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:15.109583 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.109585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.110217 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.110218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.110956 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.110958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.111320 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.111321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.111374 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.111672 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:15.111676 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:15.112908 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:15.114570 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.114571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.114572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.114773 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.114964 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.114965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.114965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.115028 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.115264 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:15.115265 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:15.115298 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 15:09:15.115341 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:09:15.115417 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260807 15:09:15.115579 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.109296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51090 (local address 127.18.129.60:46623)
0807 15:09:15.109328 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.109330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.109337 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:09:15.109379 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.109380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.109388 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.109428 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.109647 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.110141 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.111033 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.111236 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.111287 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.113021 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.113022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.113023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.113029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.114498 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.114836 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.114837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.114837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.114912 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.115093 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.115093 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.115094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.115210 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.115270 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.115375 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.115474 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":19}
I20260807 15:09:15.116433 6762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } attempt: 2
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:09:15.117167 6989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005: Bootstrap starting.
I20260807 15:09:15.118243 6989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:15.118562 6989 log.cc:824] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:15.119483 6989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005: No bootstrap required, opened a new log
I20260807 15:09:15.120010 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.113046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40984 (local address 127.18.129.61:38795)
0807 15:09:15.113086 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.113088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.113125 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:09:15.113166 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.113167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.113176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.113239 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.113615 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.114272 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.115069 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.115335 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.115393 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.117453 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.117454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.117454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.117462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.118899 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.119236 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.119237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.119238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.119299 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.119487 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.119487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.119488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.119613 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.119813 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.119848 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.119924 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:09:15.121618 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:09:15.121740 6989 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
I20260807 15:09:15.121917 6989 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:15.121994 6989 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0516071bba4a8097911f1924d56005, State: Initialized, Role: FOLLOWER
I20260807 15:09:15.122146 6989 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:09:15.122438 7001 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.122490 6989 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:15.122577 7001 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:09:15.124745 7030 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:09:15.124859 7030 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:09:15.125972 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.116671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51096)
0807 15:09:15.116717 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.116726 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:15.116760 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:15.119030 (+ 2270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:15.119031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:15.119177 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:15.119220 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.119223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.119983 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.119984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.120591 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.120594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.120979 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.120980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.121042 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.121363 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:15.121368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:15.122765 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:15.124884 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.124885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.124885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.125090 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.125299 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.125300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.125300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.125374 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.125699 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:15.125700 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:15.125744 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 15:09:15.125788 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:09:15.125879 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:09:15.126478 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.118867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.18.129.60:46623)
0807 15:09:15.118903 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.118905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.118912 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:09:15.118930 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.118931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.118940 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.119017 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.119287 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.119891 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.120668 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.120894 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.120952 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.123272 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.123273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.123274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.123280 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.124791 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.125166 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.125167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.125168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.125225 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.125453 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.125455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.125455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.125617 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.126284 (+ 667us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.126327 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.126403 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:09:15.125567 6884 master_runner.cc:186] Error getting master registration for 127.18.129.62:43757: OK, instance_id { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" instance_seqno: 1786115354775768 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:09:15.128563 6884 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:09:15.129866 6762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc: Bootstrap starting.
I20260807 15:09:15.130625 6762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:15.130975 6762 log.cc:824] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:15.131871 6762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc: No bootstrap required, opened a new log
I20260807 15:09:15.132275 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.123194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.18.129.61:38795)
0807 15:09:15.123234 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.123236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.123618 (+ 382us) server_negotiation.cc:407] Connection header received
0807 15:09:15.124111 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.124112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.124123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.124154 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.125959 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.126631 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.127646 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.127907 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.127948 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.129410 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.129411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.129411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.129419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.130957 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.131395 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.131397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.131397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.131456 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.131682 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.131683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.131683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.131809 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.131896 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.132113 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.132192 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:09:15.133786 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.123184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.18.129.62:43757)
0807 15:09:15.123238 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.123240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.123402 (+ 162us) server_negotiation.cc:407] Connection header received
0807 15:09:15.123497 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.123498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.123508 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.123536 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.124390 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.125152 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.128127 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.128673 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.128718 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.130611 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.130613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.130613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.130620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.132543 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.132948 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.132950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.132950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.133006 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.133205 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.133206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.133206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.133333 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.133400 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.133621 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.133698 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:09:15.133873 6762 raft_consensus.cc:348] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
I20260807 15:09:15.134002 6762 raft_consensus.cc:374] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:15.134030 6762 raft_consensus.cc:729] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d16f87bf36394443884d6692ab75f9cc, State: Initialized, Role: FOLLOWER
I20260807 15:09:15.134171 6762 consensus_queue.cc:260] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
I20260807 15:09:15.134455 6762 sys_catalog.cc:564] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:15.134650 7033 sys_catalog.cc:455] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.134720 7033 sys_catalog.cc:458] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:09:15.136607 7051 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:09:15.136708 7051 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:09:15.139353 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.123056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.18.129.60:46623)
0807 15:09:15.123362 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.123363 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.128328 (+ 4965us) server_negotiation.cc:407] Connection header received
0807 15:09:15.128444 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.128445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.128455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.128499 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.132777 (+ 4278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.133566 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.134558 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.134929 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.134994 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.136364 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.136366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.136366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.136374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.138029 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.138398 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.138399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.138400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.138453 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.138726 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.138727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.138727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.138870 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.138940 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.139150 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.139231 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":63,"threads_started":1}
I20260807 15:09:15.137194 6663 master_runner.cc:186] Error getting master registration for 127.18.129.61:38795: OK, instance_id { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" instance_seqno: 1786115354935612 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:09:15.139482 6663 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:09:15.170022 6877 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } attempt: 3
Aug 07 15:09:15 dist-test-slave-5bn2 krb5kdc[6642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115354, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:09:15.178555 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.170409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.18.129.60:46623)
0807 15:09:15.170465 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:15.170467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:15.170476 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:09:15.170498 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:15.170499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:15.170509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:15.170570 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:15.170945 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.171724 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.172658 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.173019 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.173088 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.175728 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:15.175729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:15.175729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:15.175737 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:15.177380 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.177775 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.177775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.177776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.177841 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:15.178021 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:15.178022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:15.178022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:15.178162 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:15.178227 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:15.178371 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:09:15.178458 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:09:15.178550 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:15.170337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:46623 (local address 127.0.0.1:51108)
0807 15:09:15.170387 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:15.170398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:15.170434 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:15.170592 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:15.170593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:15.170783 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:15.170843 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.170874 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.171805 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.171807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.172577 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:15.172580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:15.173108 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:15.173109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:15.173173 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:15.173711 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:15.173716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:15.175630 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:15.177466 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.177467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.177468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.177701 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.177901 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:15.177902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:15.177902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:15.177966 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:15.178223 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:15.178223 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:15.178271 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 15:09:15.178340 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:09:15.178441 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:09:15.180691 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc: Bootstrap starting.
I20260807 15:09:15.181468 6877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:15.181799 6877 log.cc:824] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:15.182693 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc: No bootstrap required, opened a new log
I20260807 15:09:15.184681 6877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
I20260807 15:09:15.184823 6877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:15.184852 6877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb13c82184b4cbabcf6b14c3d83eabc, State: Initialized, Role: FOLLOWER
I20260807 15:09:15.185019 6877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
I20260807 15:09:15.185331 6877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:09:15.187307 7071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:09:15.187412 7071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:09:15.185292 7055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.189157 7055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:09:15.189565 6773 master_runner.cc:263] No leader master found from master 5fb13c82184b4cbabcf6b14c3d83eabc
I20260807 15:09:15.189618 6773 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 15:09:15.273648 6689 tablet.cc:2369] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:15.306943 7033 raft_consensus.cc:482] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:15.307076 7033 raft_consensus.cc:504] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
I20260807 15:09:15.307303 7033 leader_election.cc:288] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fb13c82184b4cbabcf6b14c3d83eabc (127.18.129.61:38795), fc0516071bba4a8097911f1924d56005 (127.18.129.60:46623)
I20260807 15:09:15.307538 6946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d16f87bf36394443884d6692ab75f9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0516071bba4a8097911f1924d56005" is_pre_election: true
I20260807 15:09:15.307716 6946 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d16f87bf36394443884d6692ab75f9cc in term 0.
I20260807 15:09:15.307792 6845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d16f87bf36394443884d6692ab75f9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" is_pre_election: true
I20260807 15:09:15.307934 6845 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d16f87bf36394443884d6692ab75f9cc in term 0.
I20260807 15:09:15.307951 6683 leader_election.cc:302] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d16f87bf36394443884d6692ab75f9cc, fc0516071bba4a8097911f1924d56005; no voters:
I20260807 15:09:15.308153 7033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:15.308202 7033 raft_consensus.cc:482] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:15.308216 7033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:15.308897 7033 raft_consensus.cc:504] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
I20260807 15:09:15.309002 7033 leader_election.cc:288] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [CANDIDATE]: Term 1 election: Requested vote from peers 5fb13c82184b4cbabcf6b14c3d83eabc (127.18.129.61:38795), fc0516071bba4a8097911f1924d56005 (127.18.129.60:46623)
I20260807 15:09:15.309185 6845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d16f87bf36394443884d6692ab75f9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc"
I20260807 15:09:15.309224 6946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d16f87bf36394443884d6692ab75f9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0516071bba4a8097911f1924d56005"
I20260807 15:09:15.309281 6845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:15.309293 6946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:15.309942 6946 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d16f87bf36394443884d6692ab75f9cc in term 1.
I20260807 15:09:15.310110 6683 leader_election.cc:302] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d16f87bf36394443884d6692ab75f9cc, fc0516071bba4a8097911f1924d56005; no voters:
I20260807 15:09:15.310182 7033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:15.310214 6845 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d16f87bf36394443884d6692ab75f9cc in term 1.
I20260807 15:09:15.310287 7033 raft_consensus.cc:686] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [term 1 LEADER]: Becoming Leader. State: Replica: d16f87bf36394443884d6692ab75f9cc, State: Running, Role: LEADER
I20260807 15:09:15.310374 7033 consensus_queue.cc:237] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } }
I20260807 15:09:15.310813 7073 sys_catalog.cc:455] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d16f87bf36394443884d6692ab75f9cc. Latest consensus state: current_term: 1 leader_uuid: "d16f87bf36394443884d6692ab75f9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.310935 7073 sys_catalog.cc:458] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:15.311174 7077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:15.311369 7077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:15.312677 6946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [term 1 FOLLOWER]: Refusing update from remote peer d16f87bf36394443884d6692ab75f9cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:15.312913 6845 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [term 1 FOLLOWER]: Refusing update from remote peer d16f87bf36394443884d6692ab75f9cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:15.312976 7073 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:15.313172 7073 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:15.314220 7001 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d16f87bf36394443884d6692ab75f9cc. Latest consensus state: current_term: 1 leader_uuid: "d16f87bf36394443884d6692ab75f9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.314216 7055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d16f87bf36394443884d6692ab75f9cc. Latest consensus state: current_term: 1 leader_uuid: "d16f87bf36394443884d6692ab75f9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.314297 7055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:09:15.314297 7001 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:09:15.315240 7033 sys_catalog.cc:455] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d16f87bf36394443884d6692ab75f9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.315318 7033 sys_catalog.cc:458] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:15.315407 7033 sys_catalog.cc:455] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d16f87bf36394443884d6692ab75f9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.315455 7033 sys_catalog.cc:458] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:15.315748 7077 catalog_manager.cc:1347] Generated new cluster ID: e2b5c43a4f794a28b542e7ff08fae41f
I20260807 15:09:15.315733 7055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d16f87bf36394443884d6692ab75f9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.315802 7055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:09:15.315935 7001 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d16f87bf36394443884d6692ab75f9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d16f87bf36394443884d6692ab75f9cc" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43757 } } peers { permanent_uuid: "5fb13c82184b4cbabcf6b14c3d83eabc" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38795 } } peers { permanent_uuid: "fc0516071bba4a8097911f1924d56005" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 46623 } } }
I20260807 15:09:15.315968 7080 mvcc.cc:205] Tried to move back new op lower bound from 7315928495358672896 to 7315928495351869440. Current Snapshot: MvccSnapshot[applied={T|T < 7315928495358672896}]
I20260807 15:09:15.315968 7081 mvcc.cc:205] Tried to move back new op lower bound from 7315928495358672896 to 7315928495351869440. Current Snapshot: MvccSnapshot[applied={T|T < 7315928495358672896}]
I20260807 15:09:15.316011 7001 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:09:15.315912 7077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:15.332238 7077 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:15.332929 7077 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:15.337023 7077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d16f87bf36394443884d6692ab75f9cc: Generated new TSK 0
I20260807 15:09:15.337255 7077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:09:16.125366 7030 catalog_manager.cc:1259] Loaded cluster ID: e2b5c43a4f794a28b542e7ff08fae41f
I20260807 15:09:16.125455 7030 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005: loading cluster ID for follower catalog manager: success
I20260807 15:09:16.126302 7030 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005: acquiring CA information for follower catalog manager: success
I20260807 15:09:16.126603 7030 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fc0516071bba4a8097911f1924d56005: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:09:16.187875 7071 catalog_manager.cc:1259] Loaded cluster ID: e2b5c43a4f794a28b542e7ff08fae41f
I20260807 15:09:16.187958 7071 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc: loading cluster ID for follower catalog manager: success
I20260807 15:09:16.188776 7071 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc: acquiring CA information for follower catalog manager: success
I20260807 15:09:16.189090 7071 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5fb13c82184b4cbabcf6b14c3d83eabc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:09:16.392431 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 6663
I20260807 15:09:16.407418 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 6773
I20260807 15:09:16.417224 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 6884
2026-08-07T15:09:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3016 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:16 dist-test-slave-5bn2 krb5kdc[7103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:16 dist-test-slave-5bn2 krb5kdc[7103](info): set up 2 sockets
Aug 07 15:09:16 dist-test-slave-5bn2 krb5kdc[7103](info): commencing operation
krb5kdc: starting...
W20260807 15:09:17.553777 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:09:17 dist-test-slave-5bn2 krb5kdc[7103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:09:17.675172 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34369
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:34369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:17.758062 7128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:17.758272 7128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:17.758293 7128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:17.759500 7128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:17.759544 7128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:17.759557 7128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:17.759568 7128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:17.759579 7128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:17.761090 7128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:34369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:17.761315 7128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:17.761524 7128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:17.766448 7135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:17.766517 7134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:17.766455 7142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:17.766788 7128 server_base.cc:1034] running on GCE node
I20260807 15:09:17.767117 7128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:17.767406 7128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:17.768551 7128 hybrid_clock.cc:648] HybridClock initialized: now 1786115357768534 us; error 29 us; skew 500 ppm
Aug 07 15:09:17 dist-test-slave-5bn2 krb5kdc[7103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115357, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:17.770996 7128 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:17.771454 7128 webserver.cc:460] Webserver started at http://127.18.129.62:35005/ using document root <none> and password file <none>
I20260807 15:09:17.771659 7128 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:17.771708 7128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:17.771840 7128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:17.772888 7128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "300e7b7b11ae45409b419e9f98080549"
format_stamp: "Formatted at 2026-08-07 15:09:17 on dist-test-slave-5bn2"
server_key: "9474f703afbfa4c8d69985f1c6574ac4"
server_key_iv: "b3a68283f79862057ab831f363ab8c26"
server_key_version: "encryptionkey@0"
I20260807 15:09:17.773253 7128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "300e7b7b11ae45409b419e9f98080549"
format_stamp: "Formatted at 2026-08-07 15:09:17 on dist-test-slave-5bn2"
server_key: "9474f703afbfa4c8d69985f1c6574ac4"
server_key_iv: "b3a68283f79862057ab831f363ab8c26"
server_key_version: "encryptionkey@0"
I20260807 15:09:17.774513 7128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:09:17.775218 7153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:17.775370 7128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:17.775458 7128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "300e7b7b11ae45409b419e9f98080549"
format_stamp: "Formatted at 2026-08-07 15:09:17 on dist-test-slave-5bn2"
server_key: "9474f703afbfa4c8d69985f1c6574ac4"
server_key_iv: "b3a68283f79862057ab831f363ab8c26"
server_key_version: "encryptionkey@0"
I20260807 15:09:17.775520 7128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:17.786900 7128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:17.788583 7128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:17.788715 7128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:17.792725 7128 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:34369
I20260807 15:09:17.792762 7229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:34369 every 8 connection(s)
I20260807 15:09:17.793092 7128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:17.793931 7231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:17.796259 7231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549: Bootstrap starting.
I20260807 15:09:17.796797 7231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:17.797041 7231 log.cc:824] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:17.797695 7231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549: No bootstrap required, opened a new log
I20260807 15:09:17.799263 7231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300e7b7b11ae45409b419e9f98080549" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34369 } }
I20260807 15:09:17.799399 7231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:17.799420 7231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300e7b7b11ae45409b419e9f98080549, State: Initialized, Role: FOLLOWER
I20260807 15:09:17.799533 7231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300e7b7b11ae45409b419e9f98080549" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34369 } }
I20260807 15:09:17.799618 7231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:17.799642 7231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:17.799670 7231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:17.800160 7231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300e7b7b11ae45409b419e9f98080549" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34369 } }
I20260807 15:09:17.800246 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 7128
I20260807 15:09:17.800274 7231 leader_election.cc:302] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 300e7b7b11ae45409b419e9f98080549; no voters:
I20260807 15:09:17.800348 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:09:17.800434 7231 leader_election.cc:288] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:17.800469 7236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:17.800541 18948 external_mini_cluster.cc:1428] Setting key be5edd2985958ee2fcb3afdbec7d60ee
I20260807 15:09:17.800658 7236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [term 1 LEADER]: Becoming Leader. State: Replica: 300e7b7b11ae45409b419e9f98080549, State: Running, Role: LEADER
I20260807 15:09:17.800712 7231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:17.800747 7236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300e7b7b11ae45409b419e9f98080549" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34369 } }
I20260807 15:09:17.801194 7242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 300e7b7b11ae45409b419e9f98080549. Latest consensus state: current_term: 1 leader_uuid: "300e7b7b11ae45409b419e9f98080549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300e7b7b11ae45409b419e9f98080549" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34369 } } }
I20260807 15:09:17.801287 7240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "300e7b7b11ae45409b419e9f98080549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300e7b7b11ae45409b419e9f98080549" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34369 } } }
I20260807 15:09:17.801432 7240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:17.801631 7242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:17.801694 7259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:17.802270 7259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:17.803722 7259 catalog_manager.cc:1347] Generated new cluster ID: 269ce7cbe2aa470ba17b2481f7d90e99
I20260807 15:09:17.804850 7259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:09:17 dist-test-slave-5bn2 krb5kdc[7103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:17.811136 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:17.801021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.18.129.62:34369)
0807 15:09:17.801429 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:17.801439 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:17.801901 (+ 462us) server_negotiation.cc:407] Connection header received
0807 15:09:17.802057 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:17.802064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:17.802123 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:17.802262 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:17.803142 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:17.804510 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:17.805396 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:17.805819 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:17.805916 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:17.808155 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:17.808159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:17.808160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:17.808170 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:17.809683 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:17.809994 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:17.809998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:17.809998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:17.810105 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:17.810281 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:17.810282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:17.810282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:17.810540 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:17.810691 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:17.810784 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 15:09:17.810941 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":85,"threads_started":1}
I20260807 15:09:17.811959 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:09:17.831722 7259 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:17.832458 7259 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:17.844760 7259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 300e7b7b11ae45409b419e9f98080549: Generated new TSK 0
I20260807 15:09:17.844974 7259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:09:17.865494 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 7128
2026-08-07T15:09:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1452 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:17 dist-test-slave-5bn2 krb5kdc[7276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:17 dist-test-slave-5bn2 krb5kdc[7276](info): set up 2 sockets
Aug 07 15:09:17 dist-test-slave-5bn2 krb5kdc[7276](info): commencing operation
krb5kdc: starting...
W20260807 15:09:19.001587 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:09:19.122134 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37941
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:37941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:19.207695 7298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:19.207981 7298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:19.208026 7298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:19.209684 7298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:19.209750 7298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:19.209770 7298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:19.209786 7298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:19.209810 7298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:19.212060 7298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:37941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:19.212337 7298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:19.212615 7298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:19.220840 7305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:19.220840 7307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:19.220999 7298 server_base.cc:1034] running on GCE node
W20260807 15:09:19.220887 7304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:19.221369 7298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:19.221735 7298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:19.222925 7298 hybrid_clock.cc:648] HybridClock initialized: now 1786115359222909 us; error 28 us; skew 500 ppm
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:19.226145 7298 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:19.226591 7298 webserver.cc:460] Webserver started at http://127.18.129.62:36019/ using document root <none> and password file <none>
I20260807 15:09:19.226811 7298 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:19.226889 7298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:19.227037 7298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:19.228205 7298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "8d42031dc2ab476f9e938ac11e7c49af"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "c69690adaa62b2df7c087020351491b1"
server_key_iv: "805d6540734168d524a97a24f6004d93"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.228549 7298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "8d42031dc2ab476f9e938ac11e7c49af"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "c69690adaa62b2df7c087020351491b1"
server_key_iv: "805d6540734168d524a97a24f6004d93"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.230063 7298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:09:19.231010 7314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.231185 7298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:19.231289 7298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "8d42031dc2ab476f9e938ac11e7c49af"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "c69690adaa62b2df7c087020351491b1"
server_key_iv: "805d6540734168d524a97a24f6004d93"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.231364 7298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:19.252184 7298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:19.254051 7298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:19.254208 7298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:19.258406 7298 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:37941
I20260807 15:09:19.258460 7394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:37941 every 8 connection(s)
I20260807 15:09:19.258786 7298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:19.259811 7395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:19.262241 7395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Bootstrap starting.
I20260807 15:09:19.262832 7395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:19.263151 7395 log.cc:824] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:19.263834 7395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: No bootstrap required, opened a new log
I20260807 15:09:19.265441 7395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:19.265590 7395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:19.265614 7395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d42031dc2ab476f9e938ac11e7c49af, State: Initialized, Role: FOLLOWER
I20260807 15:09:19.265739 7395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:19.265802 7395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:19.265825 7395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:19.265861 7395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:19.266654 7395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:19.266754 7395 leader_election.cc:302] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d42031dc2ab476f9e938ac11e7c49af; no voters:
I20260807 15:09:19.266937 7395 leader_election.cc:288] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:19.266979 7398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:19.267127 7398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 1 LEADER]: Becoming Leader. State: Replica: 8d42031dc2ab476f9e938ac11e7c49af, State: Running, Role: LEADER
I20260807 15:09:19.267205 7395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:19.267220 7398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:19.267624 7402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d42031dc2ab476f9e938ac11e7c49af. Latest consensus state: current_term: 1 leader_uuid: "8d42031dc2ab476f9e938ac11e7c49af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } } }
I20260807 15:09:19.267719 7402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:19.267730 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 7298
I20260807 15:09:19.267841 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:09:19.267867 7400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d42031dc2ab476f9e938ac11e7c49af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } } }
I20260807 15:09:19.267925 7400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:19.268035 18948 external_mini_cluster.cc:1428] Setting key ecbcba87804898f556225a0a1f3ebb9b
I20260807 15:09:19.268484 7410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:19.269282 7410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:19.270833 7410 catalog_manager.cc:1347] Generated new cluster ID: bfcdb60b65c344c7863d6d2bbe9d3d10
I20260807 15:09:19.270911 7410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:19.279820 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.268550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.18.129.62:37941)
0807 15:09:19.268704 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.268715 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.268752 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:09:19.268874 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.268882 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.269003 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.269180 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:19.270312 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.271726 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.272783 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.273416 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.273524 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.276297 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:19.276301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:19.276302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:19.276314 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:19.278221 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.278619 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.278621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.278622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.278720 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.278922 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.278922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.278923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.279211 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:19.279353 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.279453 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.279594 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260807 15:09:19.281739 7410 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:19.282225 7410 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:19.294915 7410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Generated new TSK 0
I20260807 15:09:19.295102 7410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:09:19.329816 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37941
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:19.412497 7430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:19.412698 7430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:19.412719 7430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:19.413825 7430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:19.413866 7430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:19.413893 7430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:09:19.415423 7430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:19.415653 7430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:19.415860 7430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:19.420773 7437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:19.420763 7439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:19.420913 7430 server_base.cc:1034] running on GCE node
W20260807 15:09:19.420763 7441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:19.421239 7430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:19.421470 7430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:19.422609 7430 hybrid_clock.cc:648] HybridClock initialized: now 1786115359422592 us; error 33 us; skew 500 ppm
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:19.425014 7430 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:09:19.425469 7430 webserver.cc:460] Webserver started at http://127.18.129.1:33655/ using document root <none> and password file <none>
I20260807 15:09:19.425707 7430 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:19.425757 7430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:19.425885 7430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:19.427062 7430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "aabdea75bf7b44e59552f47437d71236"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "172b10d94bf64e373cee7c35fe1c0233"
server_key_iv: "a5b1fae3b9e35afb5fbf3d1de1270b48"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.427404 7430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "aabdea75bf7b44e59552f47437d71236"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "172b10d94bf64e373cee7c35fe1c0233"
server_key_iv: "a5b1fae3b9e35afb5fbf3d1de1270b48"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.428615 7430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:09:19.429260 7452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.429412 7430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:19.429486 7430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "aabdea75bf7b44e59552f47437d71236"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "172b10d94bf64e373cee7c35fe1c0233"
server_key_iv: "a5b1fae3b9e35afb5fbf3d1de1270b48"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.429565 7430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:19.456374 7430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:19.458470 7430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:19.458647 7430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:19.458953 7430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:19.459308 7430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:19.459344 7430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.459383 7430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:19.459408 7430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.465126 7430 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:34609
I20260807 15:09:19.465195 7612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:34609 every 8 connection(s)
I20260807 15:09:19.465492 7430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:19.474637 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.466209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:55959 (local address 127.18.129.62:37941)
0807 15:09:19.466264 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.466266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.466451 (+ 185us) server_negotiation.cc:407] Connection header received
0807 15:09:19.467075 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.467077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.467089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.467136 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:19.468189 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.468832 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.469900 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.470115 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.470172 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.472242 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:19.472244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:19.472244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:19.472252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:19.473324 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.473759 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.473760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.473760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.473816 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.474037 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.474038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.474039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.474144 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:19.474201 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.474457 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.474543 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:09:19.474787 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.466172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:55959)
0807 15:09:19.466332 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.466371 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:19.466961 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:19.467215 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:19.467219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:19.467887 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:19.468097 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.468103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.468947 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.468948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.469812 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.469815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.470188 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.470189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.470320 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.470754 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:19.470761 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:19.472157 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:19.473473 (+ 1316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:19.473481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:19.473483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:19.473693 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:19.473882 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:19.473882 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:19.473883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:19.473982 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:19.474199 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:19.474207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:19.474337 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:09:19.474425 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:09:19.474577 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260807 15:09:19.475293 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 7430
I20260807 15:09:19.475324 7613 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37941
I20260807 15:09:19.475404 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:19.475415 7613 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:19.475579 18948 external_mini_cluster.cc:1428] Setting key 3d013af361dc641d16c4561fd4362819
I20260807 15:09:19.475586 7613 heartbeater.cc:511] Master 127.18.129.62:37941 requested a full tablet report, sending...
I20260807 15:09:19.476126 7335 ts_manager.cc:194] Registered new tserver with Master: aabdea75bf7b44e59552f47437d71236 (127.18.129.1:34609)
I20260807 15:09:19.476902 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:55959
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:09:19.500075 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37941
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:19.582919 7621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:19.583119 7621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:19.583149 7621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:19.584301 7621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:19.584343 7621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:19.584378 7621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:09:19.585783 7621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:19.586012 7621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:19.586215 7621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:19.591210 7631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:19.591259 7634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:19.591288 7621 server_base.cc:1034] running on GCE node
W20260807 15:09:19.591210 7637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:19.591593 7621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:19.591805 7621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:19.592940 7621 hybrid_clock.cc:648] HybridClock initialized: now 1786115359592924 us; error 33 us; skew 500 ppm
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:19.595335 7621 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:09:19.595794 7621 webserver.cc:460] Webserver started at http://127.18.129.2:36109/ using document root <none> and password file <none>
I20260807 15:09:19.596024 7621 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:19.596072 7621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:19.596167 7621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:19.597436 7621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "5879df268355427f80f25ab29fed940f"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "eeb03a8458dded5c88992e8fb11d214f"
server_key_iv: "dd288de80e0bf7b323fec85514aad350"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.597811 7621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "5879df268355427f80f25ab29fed940f"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "eeb03a8458dded5c88992e8fb11d214f"
server_key_iv: "dd288de80e0bf7b323fec85514aad350"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.599046 7621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:09:19.599674 7647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.599817 7621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.599890 7621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "5879df268355427f80f25ab29fed940f"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "eeb03a8458dded5c88992e8fb11d214f"
server_key_iv: "dd288de80e0bf7b323fec85514aad350"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.599942 7621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:19.610499 7621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:19.612239 7621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:19.612347 7621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:19.612519 7621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:19.612782 7621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:19.612815 7621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.612835 7621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:19.612846 7621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.618500 7621 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43015
I20260807 15:09:19.618559 7827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43015 every 8 connection(s)
I20260807 15:09:19.618844 7621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:19.626022 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 7621
I20260807 15:09:19.626113 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:19.626260 18948 external_mini_cluster.cc:1428] Setting key c49a10ae72f7c776a2b304a59b370b65
I20260807 15:09:19.627789 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.619533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:40213 (local address 127.18.129.62:37941)
0807 15:09:19.619574 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.619576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.619778 (+ 202us) server_negotiation.cc:407] Connection header received
0807 15:09:19.620367 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.620368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.620379 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.620451 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:19.621366 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.621918 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.622878 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.623088 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.623142 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.625077 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:19.625078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:19.625079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:19.625085 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:19.626421 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.626890 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.626891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.626892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.626949 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.627202 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.627203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.627203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.627314 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:19.627367 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.627657 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.627724 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:09:19.628014 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.619509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:40213)
0807 15:09:19.619667 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.619707 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:19.620178 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:19.620518 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:19.620521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:19.621094 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:19.621282 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.621287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.621987 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.621987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.622771 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.622773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.623154 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.623154 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.623253 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.623692 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:19.623700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:19.624981 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:19.626501 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:19.626509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:19.626511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:19.626802 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:19.627009 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:19.627009 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:19.627010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:19.627144 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:19.627374 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:19.627380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:19.627533 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 15:09:19.627641 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 15:09:19.627796 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260807 15:09:19.628495 7828 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37941
I20260807 15:09:19.628587 7828 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:19.628760 7828 heartbeater.cc:511] Master 127.18.129.62:37941 requested a full tablet report, sending...
I20260807 15:09:19.629137 7335 ts_manager.cc:194] Registered new tserver with Master: 5879df268355427f80f25ab29fed940f (127.18.129.2:43015)
I20260807 15:09:19.629488 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:40213
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:09:19.651610 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37941
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:19.732203 7835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:19.732434 7835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:19.732465 7835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:19.733596 7835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:19.733631 7835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:19.733656 7835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:09:19.735126 7835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:19.735306 7835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:19.735482 7835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:19.740334 7840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:19.740365 7841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:19.740589 7835 server_base.cc:1034] running on GCE node
W20260807 15:09:19.740769 7844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:19.740964 7835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:19.741191 7835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:19.742316 7835 hybrid_clock.cc:648] HybridClock initialized: now 1786115359742302 us; error 29 us; skew 500 ppm
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:19.748818 7835 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:09:19.749287 7835 webserver.cc:460] Webserver started at http://127.18.129.3:33927/ using document root <none> and password file <none>
I20260807 15:09:19.749485 7835 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:19.749528 7835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:19.749646 7835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:19.750670 7835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "ab7ba647706144daa018d3b6b345d120"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "86bfe2962014c17ee45236040388afb3"
server_key_iv: "99940323d1f9088b81958a5d34340e4e"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.751022 7835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "ab7ba647706144daa018d3b6b345d120"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "86bfe2962014c17ee45236040388afb3"
server_key_iv: "99940323d1f9088b81958a5d34340e4e"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.752219 7835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:19.752869 7853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.753003 7835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:09:19.753105 7835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "ab7ba647706144daa018d3b6b345d120"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "86bfe2962014c17ee45236040388afb3"
server_key_iv: "99940323d1f9088b81958a5d34340e4e"
server_key_version: "encryptionkey@0"
I20260807 15:09:19.753166 7835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:19.764725 7835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:19.766484 7835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:19.766619 7835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:19.766841 7835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:19.767210 7835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:19.767254 7835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.767283 7835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:19.767300 7835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:19.773281 7835 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:33209
I20260807 15:09:19.773335 8026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:33209 every 8 connection(s)
I20260807 15:09:19.773630 7835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:19.776407 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 7835
I20260807 15:09:19.776503 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:19.776691 18948 external_mini_cluster.cc:1428] Setting key ac95c8bc0a3eeb54ce781c2e29a28599
Aug 07 15:09:19 dist-test-slave-5bn2 krb5kdc[7276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:19.782397 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.774261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:38567 (local address 127.18.129.62:37941)
0807 15:09:19.774302 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.774304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.774496 (+ 192us) server_negotiation.cc:407] Connection header received
0807 15:09:19.775056 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.775057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.775071 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.775130 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:19.776055 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.776616 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.777915 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.778108 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.778162 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.780124 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:19.780125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:19.780126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:19.780132 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:19.781165 (+ 1033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.781513 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.781514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.781514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.781585 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.781802 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.781803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.781804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.781913 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:19.781969 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.782228 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.782308 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:09:19.782547 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.774247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:38567)
0807 15:09:19.774385 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.774420 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:19.774963 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:19.775199 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:19.775203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:19.775773 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:19.775958 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.775963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.776706 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.776707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.777833 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.777836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.778177 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.778178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.778309 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.778712 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:19.778720 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:19.780027 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:19.781235 (+ 1208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:19.781242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:19.781244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:19.781447 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:19.781641 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:19.781642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:19.781642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:19.781748 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:19.781968 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:19.781975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:19.782112 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 15:09:19.782199 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:09:19.782343 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
I20260807 15:09:19.782938 8027 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37941
I20260807 15:09:19.783021 8027 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:19.783167 8027 heartbeater.cc:511] Master 127.18.129.62:37941 requested a full tablet report, sending...
I20260807 15:09:19.783506 7335 ts_manager.cc:194] Registered new tserver with Master: ab7ba647706144daa018d3b6b345d120 (127.18.129.3:33209)
I20260807 15:09:19.783838 7335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:38567
I20260807 15:09:19.788184 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:19.794555 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.789420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.18.129.62:37941)
0807 15:09:19.789472 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.789474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.789567 (+ 93us) server_negotiation.cc:407] Connection header received
0807 15:09:19.789624 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.789624 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.789632 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.789677 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:19.790175 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.790630 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.791306 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.791475 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.791519 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.792488 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:19.792489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:19.792490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:19.792495 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:19.793729 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.793972 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.793973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.793973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.794016 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:19.794138 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:19.794138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:19.794138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:19.794268 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:19.794335 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.794408 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.794464 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:09:19.795989 7335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:19.797542 7335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:19.803809 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.799734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:33209 (local address 127.0.0.1:44486)
0807 15:09:19.799839 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.799879 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:19.799941 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:19.800239 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:19.800243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:19.800245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:19.800310 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.800312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.801387 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.801388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.802598 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.802601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.803518 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.803519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.803605 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.803619 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:09:19.803683 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:09:19.803733 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":63,"threads_started":1}
I20260807 15:09:19.803920 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.799736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43015 (local address 127.0.0.1:37668)
0807 15:09:19.799855 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.799885 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:19.799942 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:19.800458 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:19.800460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:19.800462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:19.800516 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.800518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.801859 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.801860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.802881 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.802884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.803684 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.803686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.803749 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.803753 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:19.803806 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:09:19.803845 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":78,"threads_started":1}
I20260807 15:09:19.804068 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.800101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37668 (local address 127.18.129.2:43015)
0807 15:09:19.800283 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.800287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.800297 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:19.800335 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.800337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.800352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.800590 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:19.800670 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.801744 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.802947 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.803609 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.803674 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.803731 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.803821 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.803939 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260807 15:09:19.804219 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.799940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34609 (local address 127.0.0.1:38784)
0807 15:09:19.800367 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.800376 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:19.800587 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:19.800883 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:19.800884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:19.800886 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:19.800924 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.800925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.802067 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.802068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.803047 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.803050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.803993 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.803994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.804061 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.804065 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:19.804118 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:09:19.804155 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":73,"threads_started":1}
I20260807 15:09:19.804267 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.799912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.18.129.3:33209)
0807 15:09:19.800085 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.800089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.800097 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:09:19.800134 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.800136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.800147 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.800218 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:19.800406 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.801302 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.802682 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.803371 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.803434 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.803484 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.804163 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.804201 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260807 15:09:19.804504 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.800379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.18.129.1:34609)
0807 15:09:19.800733 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.800737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.800747 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:19.800782 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.800784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.800797 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.800876 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:19.800979 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.801978 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.803130 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.803828 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.803890 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.803959 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.804380 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.804426 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":88,"threads_started":1}
I20260807 15:09:19.804747 7723 tablet_service.cc:1467] Processing CreateTablet for tablet 865b22ca6b3143489c40fb5580bd3c7a (DEFAULT_TABLE table=test-table [id=959ab4f6dff94adc92b16c9c87ef90d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:19.804884 7924 tablet_service.cc:1467] Processing CreateTablet for tablet 865b22ca6b3143489c40fb5580bd3c7a (DEFAULT_TABLE table=test-table [id=959ab4f6dff94adc92b16c9c87ef90d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:19.805038 7517 tablet_service.cc:1467] Processing CreateTablet for tablet 865b22ca6b3143489c40fb5580bd3c7a (DEFAULT_TABLE table=test-table [id=959ab4f6dff94adc92b16c9c87ef90d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:19.805147 7723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865b22ca6b3143489c40fb5580bd3c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:19.805231 7924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865b22ca6b3143489c40fb5580bd3c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:19.805341 7517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 865b22ca6b3143489c40fb5580bd3c7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:19.808107 8050 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Bootstrap starting.
I20260807 15:09:19.808459 8051 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: Bootstrap starting.
I20260807 15:09:19.808621 8052 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Bootstrap starting.
I20260807 15:09:19.808741 8050 tablet_bootstrap.cc:654] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:19.809177 8052 tablet_bootstrap.cc:654] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:19.809185 8051 tablet_bootstrap.cc:654] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:19.809357 8050 log.cc:824] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:19.809654 8052 log.cc:824] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:19.809708 8051 log.cc:824] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:19.810257 8050 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: No bootstrap required, opened a new log
I20260807 15:09:19.810314 8050 ts_tablet_manager.cc:1397] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:09:19.810345 8052 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: No bootstrap required, opened a new log
I20260807 15:09:19.810410 8052 ts_tablet_manager.cc:1397] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:19.811266 8051 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: No bootstrap required, opened a new log
I20260807 15:09:19.811336 8051 ts_tablet_manager.cc:1397] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:19.811945 8052 raft_consensus.cc:348] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.812086 8052 raft_consensus.cc:374] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:19.812127 8052 raft_consensus.cc:729] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aabdea75bf7b44e59552f47437d71236, State: Initialized, Role: FOLLOWER
I20260807 15:09:19.812131 8050 raft_consensus.cc:348] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.812235 8052 consensus_queue.cc:260] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.812294 8050 raft_consensus.cc:374] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:19.812336 8050 raft_consensus.cc:729] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5879df268355427f80f25ab29fed940f, State: Initialized, Role: FOLLOWER
I20260807 15:09:19.812433 8050 consensus_queue.cc:260] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.812484 8052 ts_tablet_manager.cc:1428] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:09:19.812618 7613 heartbeater.cc:503] Master 127.18.129.62:37941 was elected leader, sending a full tablet report...
I20260807 15:09:19.812726 8050 ts_tablet_manager.cc:1428] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:09:19.812788 7828 heartbeater.cc:503] Master 127.18.129.62:37941 was elected leader, sending a full tablet report...
I20260807 15:09:19.813484 8051 raft_consensus.cc:348] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.813630 8051 raft_consensus.cc:374] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:19.813668 8051 raft_consensus.cc:729] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7ba647706144daa018d3b6b345d120, State: Initialized, Role: FOLLOWER
I20260807 15:09:19.813791 8051 consensus_queue.cc:260] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.814018 8051 ts_tablet_manager.cc:1428] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:19.814067 8027 heartbeater.cc:503] Master 127.18.129.62:37941 was elected leader, sending a full tablet report...
W20260807 15:09:19.869392 7829 tablet.cc:2369] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:19.903072 8057 raft_consensus.cc:482] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:19.903201 8057 raft_consensus.cc:504] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.903625 8057 leader_election.cc:288] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5879df268355427f80f25ab29fed940f (127.18.129.2:43015), ab7ba647706144daa018d3b6b345d120 (127.18.129.3:33209)
I20260807 15:09:19.906404 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.903869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:37987 (local address 127.18.129.3:33209)
0807 15:09:19.903926 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.903928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.904026 (+ 98us) server_negotiation.cc:407] Connection header received
0807 15:09:19.904087 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.904089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.904099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.904171 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:19.904327 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.904781 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.905704 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.906082 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.906143 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.906153 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.906289 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.906330 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:09:19.906424 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.903767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43015 (local address 127.18.129.1:34263)
0807 15:09:19.903824 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.903836 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:19.903879 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:19.904112 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:19.904113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:19.904116 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:19.904197 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.904200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.904845 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.904846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.905602 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.905604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.906166 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.906167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.906254 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.906257 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:09:19.906311 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:09:19.906345 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":1}
I20260807 15:09:19.906430 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.903927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:34263 (local address 127.18.129.2:43015)
0807 15:09:19.903981 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.903984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.903993 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:09:19.904024 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.904024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.904037 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.904091 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:19.904275 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.904761 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.905699 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.906086 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.906150 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.906159 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.906329 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.906368 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:09:19.906431 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.903767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:33209 (local address 127.18.129.1:37987)
0807 15:09:19.903960 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.903968 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:19.904032 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:19.904178 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:19.904178 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:19.904180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:19.904260 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.904262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.904865 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.904866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.905613 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:19.905615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.906163 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.906164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:19.906226 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.906230 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:19.906270 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:09:19.906307 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":16,"thread_start_us":115,"threads_started":1}
I20260807 15:09:19.906747 7752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865b22ca6b3143489c40fb5580bd3c7a" candidate_uuid: "aabdea75bf7b44e59552f47437d71236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5879df268355427f80f25ab29fed940f" is_pre_election: true
I20260807 15:09:19.906807 7957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865b22ca6b3143489c40fb5580bd3c7a" candidate_uuid: "aabdea75bf7b44e59552f47437d71236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab7ba647706144daa018d3b6b345d120" is_pre_election: true
I20260807 15:09:19.906926 7752 raft_consensus.cc:2454] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aabdea75bf7b44e59552f47437d71236 in term 0.
I20260807 15:09:19.906998 7957 raft_consensus.cc:2454] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aabdea75bf7b44e59552f47437d71236 in term 0.
I20260807 15:09:19.907151 7457 leader_election.cc:302] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5879df268355427f80f25ab29fed940f, aabdea75bf7b44e59552f47437d71236; no voters:
I20260807 15:09:19.907274 8057 raft_consensus.cc:2781] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:19.907331 8057 raft_consensus.cc:482] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:19.907347 8057 raft_consensus.cc:3037] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:19.908054 8057 raft_consensus.cc:504] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.908190 8057 leader_election.cc:288] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [CANDIDATE]: Term 1 election: Requested vote from peers 5879df268355427f80f25ab29fed940f (127.18.129.2:43015), ab7ba647706144daa018d3b6b345d120 (127.18.129.3:33209)
I20260807 15:09:19.908337 7752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865b22ca6b3143489c40fb5580bd3c7a" candidate_uuid: "aabdea75bf7b44e59552f47437d71236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5879df268355427f80f25ab29fed940f"
I20260807 15:09:19.908404 7957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "865b22ca6b3143489c40fb5580bd3c7a" candidate_uuid: "aabdea75bf7b44e59552f47437d71236" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab7ba647706144daa018d3b6b345d120"
I20260807 15:09:19.908427 7752 raft_consensus.cc:3037] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:19.908476 7957 raft_consensus.cc:3037] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:19.909327 7957 raft_consensus.cc:2454] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aabdea75bf7b44e59552f47437d71236 in term 1.
I20260807 15:09:19.909327 7752 raft_consensus.cc:2454] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aabdea75bf7b44e59552f47437d71236 in term 1.
I20260807 15:09:19.909493 7454 leader_election.cc:302] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aabdea75bf7b44e59552f47437d71236, ab7ba647706144daa018d3b6b345d120; no voters:
I20260807 15:09:19.909602 8057 raft_consensus.cc:2781] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:19.909754 8057 raft_consensus.cc:686] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 1 LEADER]: Becoming Leader. State: Replica: aabdea75bf7b44e59552f47437d71236, State: Running, Role: LEADER
I20260807 15:09:19.909834 8057 consensus_queue.cc:237] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:19.910650 7336 catalog_manager.cc:5495] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 reported cstate change: term changed from 0 to 1, leader changed from <none> to aabdea75bf7b44e59552f47437d71236 (127.18.129.1). New cstate: current_term: 1 leader_uuid: "aabdea75bf7b44e59552f47437d71236" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } health_report { overall_health: HEALTHY } } }
I20260807 15:09:19.917179 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.915000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.18.129.1:34609)
0807 15:09:19.915047 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:19.915049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:19.915058 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:09:19.915122 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:19.915123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:19.915141 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:19.915200 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:19.915331 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.915779 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.916481 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:19.916683 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:19.916736 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:19.916890 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:19.916984 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:19.917051 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:19.917073 (+ 22us) server_negotiation.cc:299] Negotiation successful
0807 15:09:19.917094 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:09:19.918718 7752 raft_consensus.cc:1261] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 1 FOLLOWER]: Refusing update from remote peer aabdea75bf7b44e59552f47437d71236: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:19.918720 7957 raft_consensus.cc:1261] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 1 FOLLOWER]: Refusing update from remote peer aabdea75bf7b44e59552f47437d71236: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:19.919029 8062 consensus_queue.cc:1035] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:19.919155 8057 consensus_queue.cc:1035] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:19.921895 8066 mvcc.cc:205] Tried to move back new op lower bound from 7315928514224840704 to 7315928514191187968. Current Snapshot: MvccSnapshot[applied={T|T < 7315928514224840704}]
I20260807 15:09:19.924199 7335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:19.924283 7335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:19.925026 7335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Sending DeleteTablet for 3 replicas of tablet 865b22ca6b3143489c40fb5580bd3c7a
I20260807 15:09:19.925348 7924 tablet_service.cc:1514] Processing DeleteTablet for tablet 865b22ca6b3143489c40fb5580bd3c7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:19 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:44486
I20260807 15:09:19.925395 7723 tablet_service.cc:1514] Processing DeleteTablet for tablet 865b22ca6b3143489c40fb5580bd3c7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:19 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:37668
I20260807 15:09:19.925580 8088 tablet_replica.cc:330] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: stopping tablet replica
I20260807 15:09:19.925565 7517 tablet_service.cc:1514] Processing DeleteTablet for tablet 865b22ca6b3143489c40fb5580bd3c7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:19 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:38784
I20260807 15:09:19.925681 8088 raft_consensus.cc:2229] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:19.925782 8088 raft_consensus.cc:2258] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:19.925772 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 7430
I20260807 15:09:19.925858 8089 tablet_replica.cc:330] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: stopping tablet replica
I20260807 15:09:19.926020 8089 raft_consensus.cc:2229] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:19.936009 8088 ts_tablet_manager.cc:1905] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:19.937739 8085 tablet_replica.cc:330] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: stopping tablet replica
I20260807 15:09:19.937741 8088 ts_tablet_manager.cc:1918] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:19.937816 8088 log.cc:1196] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/865b22ca6b3143489c40fb5580bd3c7a
I20260807 15:09:19.937872 8085 raft_consensus.cc:2229] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:19.937994 8085 raft_consensus.cc:2258] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:19.938081 8088 ts_tablet_manager.cc:1939] T 865b22ca6b3143489c40fb5580bd3c7a P 5879df268355427f80f25ab29fed940f: Deleting consensus metadata
I20260807 15:09:19.938494 7319 catalog_manager.cc:4841] TS 5879df268355427f80f25ab29fed940f (127.18.129.2:43015): tablet 865b22ca6b3143489c40fb5580bd3c7a (table test-table [id=959ab4f6dff94adc92b16c9c87ef90d5]) successfully deleted
I20260807 15:09:19.939385 8085 ts_tablet_manager.cc:1905] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:19.940877 8085 ts_tablet_manager.cc:1918] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:19.940959 8085 log.cc:1196] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/865b22ca6b3143489c40fb5580bd3c7a
I20260807 15:09:19.941262 8085 ts_tablet_manager.cc:1939] T 865b22ca6b3143489c40fb5580bd3c7a P ab7ba647706144daa018d3b6b345d120: Deleting consensus metadata
I20260807 15:09:19.941700 7317 catalog_manager.cc:4841] TS ab7ba647706144daa018d3b6b345d120 (127.18.129.3:33209): tablet 865b22ca6b3143489c40fb5580bd3c7a (table test-table [id=959ab4f6dff94adc92b16c9c87ef90d5]) successfully deleted
W20260807 15:09:19.943544 7317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:34609 (error 108)
I20260807 15:09:19.943845 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 7621
I20260807 15:09:19.944078 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:19.943950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34609 (local address 127.0.0.1:38806)
0807 15:09:19.943999 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:19.944030 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:09:19.951002 7317 catalog_manager.cc:4584] TS aabdea75bf7b44e59552f47437d71236 (127.18.129.1:34609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 865b22ca6b3143489c40fb5580bd3c7a: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34609: connect: Connection refused (error 111)
I20260807 15:09:19.955492 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 7835
I20260807 15:09:19.964330 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 7298
I20260807 15:09:19.972476 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37941
--webserver_interface=127.18.129.62
--webserver_port=36019
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:37941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:20.052955 8092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:20.053154 8092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:20.053174 8092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:20.054306 8092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:20.054348 8092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:20.054363 8092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:20.054384 8092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:20.054409 8092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:20.055902 8092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:37941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=36019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:20.056126 8092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:20.056352 8092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:20.065985 8101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.065979 8102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.065976 8104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.066556 8092 server_base.cc:1034] running on GCE node
I20260807 15:09:20.066843 8092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:20.067196 8092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:20.068341 8092 hybrid_clock.cc:648] HybridClock initialized: now 1786115360068326 us; error 27 us; skew 500 ppm
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:20.070724 8092 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:20.071211 8092 webserver.cc:460] Webserver started at http://127.18.129.62:36019/ using document root <none> and password file <none>
I20260807 15:09:20.071449 8092 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:20.071514 8092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:20.072881 8092 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:20.073479 8115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.073632 8092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:09:20.073722 8092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "8d42031dc2ab476f9e938ac11e7c49af"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "c69690adaa62b2df7c087020351491b1"
server_key_iv: "805d6540734168d524a97a24f6004d93"
server_key_version: "encryptionkey@0"
I20260807 15:09:20.074128 8092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:20.083704 8092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:20.085362 8092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:20.085491 8092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:20.089570 8092 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:37941
I20260807 15:09:20.089643 8200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:37941 every 8 connection(s)
I20260807 15:09:20.090011 8092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:20.091745 8201 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:09:20.092396 8201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Bootstrap starting.
I20260807 15:09:20.097565 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 8092
I20260807 15:09:20.097849 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:34609
--local_ip_for_outbound_sockets=127.18.129.1
--tserver_master_addrs=127.18.129.62:37941
--webserver_port=33655
--webserver_interface=127.18.129.1
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:20.101257 8201 log.cc:824] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:20.103303 8201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:09:20.103570 8201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Bootstrap complete.
I20260807 15:09:20.105754 8201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:20.105980 8201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d42031dc2ab476f9e938ac11e7c49af, State: Initialized, Role: FOLLOWER
I20260807 15:09:20.106140 8201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:20.106217 8201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:20.106272 8201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:20.106323 8201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:09:20.107388 8201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:20.107487 8201 leader_election.cc:302] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d42031dc2ab476f9e938ac11e7c49af; no voters:
I20260807 15:09:20.107642 8201 leader_election.cc:288] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:09:20.107700 8208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:09:20.107844 8208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 2 LEADER]: Becoming Leader. State: Replica: 8d42031dc2ab476f9e938ac11e7c49af, State: Running, Role: LEADER
I20260807 15:09:20.107895 8201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:20.107936 8208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:20.108194 8208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d42031dc2ab476f9e938ac11e7c49af. Latest consensus state: current_term: 2 leader_uuid: "8d42031dc2ab476f9e938ac11e7c49af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } } }
I20260807 15:09:20.108261 8208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:20.108386 8209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8d42031dc2ab476f9e938ac11e7c49af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } } }
I20260807 15:09:20.108436 8209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:20.108726 8214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:20.109457 8214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:20.109555 8214 catalog_manager.cc:1259] Loaded cluster ID: bfcdb60b65c344c7863d6d2bbe9d3d10
I20260807 15:09:20.109611 8214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:20.110073 8214 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 15:09:20.112823 8214 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 15:09:20.114507 8214 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:20.114681 8214 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Loaded TSK: 0
I20260807 15:09:20.114915 8214 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:09:20.191296 8205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:20.191501 8205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:20.191524 8205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:20.192654 8205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:20.192698 8205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:20.192732 8205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:09:20.194150 8205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:34609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=33655
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:20.194372 8205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:20.194574 8205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:20.199522 8243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.199594 8246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.199718 8205 server_base.cc:1034] running on GCE node
W20260807 15:09:20.199522 8244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.200047 8205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:20.200268 8205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:20.201412 8205 hybrid_clock.cc:648] HybridClock initialized: now 1786115360201391 us; error 38 us; skew 500 ppm
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:20.204052 8205 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:09:20.204589 8205 webserver.cc:460] Webserver started at http://127.18.129.1:33655/ using document root <none> and password file <none>
I20260807 15:09:20.204829 8205 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:20.204905 8205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:20.206679 8205 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:09:20.207454 8255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.207621 8205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.207721 8205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "aabdea75bf7b44e59552f47437d71236"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "172b10d94bf64e373cee7c35fe1c0233"
server_key_iv: "a5b1fae3b9e35afb5fbf3d1de1270b48"
server_key_version: "encryptionkey@0"
I20260807 15:09:20.208163 8205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:20.241830 8205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:20.244263 8205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:20.244441 8205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:20.244728 8205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:20.245239 8268 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:09:20.246711 8205 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:09:20.246780 8205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:09:20.246817 8205 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:09:20.247761 8205 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:09:20.247808 8205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:09:20.247860 8268 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Bootstrap starting.
I20260807 15:09:20.254959 8205 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:34609
I20260807 15:09:20.255383 8205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:20.257606 8445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:34609 every 8 connection(s)
I20260807 15:09:20.258037 8268 log.cc:824] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:20.260447 8268 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:09:20.260744 8268 tablet_bootstrap.cc:492] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Bootstrap complete.
I20260807 15:09:20.261899 8268 ts_tablet_manager.cc:1397] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260807 15:09:20.263226 8268 raft_consensus.cc:348] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.263563 8268 raft_consensus.cc:729] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aabdea75bf7b44e59552f47437d71236, State: Initialized, Role: FOLLOWER
I20260807 15:09:20.263705 8268 consensus_queue.cc:260] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.263738 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 8205
I20260807 15:09:20.263984 8268 ts_tablet_manager.cc:1428] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260807 15:09:20.264016 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:43015
--local_ip_for_outbound_sockets=127.18.129.2
--tserver_master_addrs=127.18.129.62:37941
--webserver_port=36109
--webserver_interface=127.18.129.2
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:20.267935 8451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.257202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:50297 (local address 127.18.129.62:37941)
0807 15:09:20.257400 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.257411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.257597 (+ 186us) server_negotiation.cc:407] Connection header received
0807 15:09:20.258336 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.258343 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.258406 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.258564 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:20.259345 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.260254 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.261700 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.262113 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.262203 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.264714 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:20.264718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:20.264719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:20.264732 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:20.266256 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.266727 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.266729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.266729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.266816 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.267096 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.267097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.267097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.267307 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:20.267370 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.267689 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.267792 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260807 15:09:20.267987 8449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.256203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:50297)
0807 15:09:20.257420 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.257477 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.258120 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.258598 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:20.258603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:20.258983 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:20.259245 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.259251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.260393 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.260394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.261622 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.261625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.262209 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.262211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.262360 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.262913 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:20.262923 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:20.264610 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:20.266346 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:20.266354 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:20.266356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:20.266654 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:20.266889 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:20.266890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:20.266890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:20.267023 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:20.267372 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:20.267380 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:20.267543 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 15:09:20.267644 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:09:20.267807 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"thread_start_us":87,"threads_started":1}
I20260807 15:09:20.268668 8447 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37941
I20260807 15:09:20.268759 8447 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:20.268921 8447 heartbeater.cc:511] Master 127.18.129.62:37941 requested a full tablet report, sending...
I20260807 15:09:20.269574 8145 ts_manager.cc:194] Registered new tserver with Master: aabdea75bf7b44e59552f47437d71236 (127.18.129.1:34609)
I20260807 15:09:20.270764 8145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:50297
I20260807 15:09:20.271744 8447 heartbeater.cc:503] Master 127.18.129.62:37941 was elected leader, sending a full tablet report...
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.1@KRBTEST.COM
I20260807 15:09:20.278388 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.270135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34609 (local address 127.0.0.1:38814)
0807 15:09:20.270416 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.270463 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.270549 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.270741 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:20.270745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:20.271163 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:20.271227 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.271230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.272347 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.272348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.273219 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.273222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.273638 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.273639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.273717 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.274150 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:20.274155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:20.275591 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:20.277324 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:20.277332 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:20.277334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:20.277525 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:20.277731 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:20.277732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:20.277733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:20.277794 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:20.278041 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:20.278048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:20.278099 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 15:09:20.278171 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:09:20.278289 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":75,"threads_started":1}
I20260807 15:09:20.278388 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.270227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.18.129.1:34609)
0807 15:09:20.270454 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.270458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.270522 (+ 64us) server_negotiation.cc:407] Connection header received
0807 15:09:20.270635 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.270637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.270651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.270728 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:20.271313 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.272274 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.273290 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.273567 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.273616 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.275674 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:20.275683 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:20.275683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:20.275690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:20.277247 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.277599 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.277602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.277602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.277680 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.277844 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.277845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.277845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.277977 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:20.278049 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.278213 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.278295 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
I20260807 15:09:20.278731 8334 tablet_service.cc:1514] Processing DeleteTablet for tablet 865b22ca6b3143489c40fb5580bd3c7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:19 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:38814
I20260807 15:09:20.279275 8458 tablet_replica.cc:330] stopping tablet replica
I20260807 15:09:20.279385 8458 raft_consensus.cc:2229] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:20.279450 8458 raft_consensus.cc:2258] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:20.279752 8458 ts_tablet_manager.cc:1905] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:20.282060 8458 ts_tablet_manager.cc:1918] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:20.282137 8458 log.cc:1196] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/865b22ca6b3143489c40fb5580bd3c7a
I20260807 15:09:20.282357 8458 ts_tablet_manager.cc:1939] T 865b22ca6b3143489c40fb5580bd3c7a P aabdea75bf7b44e59552f47437d71236: Deleting consensus metadata
I20260807 15:09:20.282877 8121 catalog_manager.cc:4841] TS aabdea75bf7b44e59552f47437d71236 (127.18.129.1:34609): tablet 865b22ca6b3143489c40fb5580bd3c7a (table test-table [id=959ab4f6dff94adc92b16c9c87ef90d5]) successfully deleted
W20260807 15:09:20.348845 8454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:20.349056 8454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:20.349079 8454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:20.350450 8454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:20.350507 8454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:20.350538 8454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:09:20.352187 8454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:43015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=36109
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:20.352415 8454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:20.352625 8454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:20.358042 8469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.358038 8467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.358062 8466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.358325 8454 server_base.cc:1034] running on GCE node
I20260807 15:09:20.358496 8454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:20.358716 8454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:20.359851 8454 hybrid_clock.cc:648] HybridClock initialized: now 1786115360359835 us; error 31 us; skew 500 ppm
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:20.362226 8454 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:09:20.362648 8454 webserver.cc:460] Webserver started at http://127.18.129.2:36109/ using document root <none> and password file <none>
I20260807 15:09:20.362841 8454 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:20.362946 8454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:20.364218 8454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:20.364756 8478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.364905 8454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:20.365000 8454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "5879df268355427f80f25ab29fed940f"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "eeb03a8458dded5c88992e8fb11d214f"
server_key_iv: "dd288de80e0bf7b323fec85514aad350"
server_key_version: "encryptionkey@0"
I20260807 15:09:20.365334 8454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:20.373028 8454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:20.375141 8454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:20.375270 8454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:20.375478 8454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:20.375756 8454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:20.375788 8454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.375816 8454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:20.375840 8454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.381575 8454 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43015
I20260807 15:09:20.381628 8655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43015 every 8 connection(s)
I20260807 15:09:20.381949 8454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:20.390007 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 8454
I20260807 15:09:20.390288 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:33209
--local_ip_for_outbound_sockets=127.18.129.3
--tserver_master_addrs=127.18.129.62:37941
--webserver_port=33927
--webserver_interface=127.18.129.3
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:20.391520 8451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.382769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:38377 (local address 127.18.129.62:37941)
0807 15:09:20.382834 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.382836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.383089 (+ 253us) server_negotiation.cc:407] Connection header received
0807 15:09:20.383691 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.383693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.383713 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.383773 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:20.384764 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.385292 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.386310 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.386534 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.386600 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.388912 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:20.388914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:20.388914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:20.388922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:20.390170 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.390607 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.390608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.390608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.390666 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.390931 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.390931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.390931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.391031 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:20.391093 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.391379 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.391452 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:09:20.391723 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.382737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:38377)
0807 15:09:20.382963 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.383000 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.383576 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.383845 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:20.383850 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:20.384454 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:20.384657 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.384663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.385394 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.385395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.386216 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.386219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.386615 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.386616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.386744 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.387229 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:20.387238 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:20.388805 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:20.390249 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:20.390257 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:20.390259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:20.390537 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:20.390721 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:20.390721 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:20.390722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:20.390878 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:20.391099 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:20.391104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:20.391259 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 15:09:20.391357 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:09:20.391512 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":133,"threads_started":1}
I20260807 15:09:20.392197 8659 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37941
I20260807 15:09:20.392304 8659 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:20.392488 8659 heartbeater.cc:511] Master 127.18.129.62:37941 requested a full tablet report, sending...
I20260807 15:09:20.392882 8145 ts_manager.cc:194] Registered new tserver with Master: 5879df268355427f80f25ab29fed940f (127.18.129.2:43015)
I20260807 15:09:20.393240 8145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:38377
W20260807 15:09:20.474797 8662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:20.475050 8662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:20.475072 8662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:20.476222 8662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:20.476261 8662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:20.476287 8662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:09:20.477896 8662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:33209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=33927
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:20.478143 8662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:20.478372 8662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:20.483381 8673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.483412 8671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.483433 8677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.483630 8662 server_base.cc:1034] running on GCE node
I20260807 15:09:20.483814 8662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:20.484038 8662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:20.485193 8662 hybrid_clock.cc:648] HybridClock initialized: now 1786115360485177 us; error 32 us; skew 500 ppm
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:20.487730 8662 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:09:20.488148 8662 webserver.cc:460] Webserver started at http://127.18.129.3:33927/ using document root <none> and password file <none>
I20260807 15:09:20.488370 8662 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:20.488436 8662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:20.489820 8662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:20.490408 8691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.490585 8662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:09:20.490672 8662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "ab7ba647706144daa018d3b6b345d120"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "86bfe2962014c17ee45236040388afb3"
server_key_iv: "99940323d1f9088b81958a5d34340e4e"
server_key_version: "encryptionkey@0"
I20260807 15:09:20.491065 8662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:20.502244 8662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:20.504110 8662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:20.504251 8662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:20.504464 8662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:20.504763 8662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:20.504797 8662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.504827 8662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:20.504850 8662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.510818 8662 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:33209
I20260807 15:09:20.510941 8868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:33209 every 8 connection(s)
I20260807 15:09:20.511229 8662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:20.516518 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 8662
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:20.520735 8451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.511959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:45783 (local address 127.18.129.62:37941)
0807 15:09:20.512025 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.512027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.512241 (+ 214us) server_negotiation.cc:407] Connection header received
0807 15:09:20.512756 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.512757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.512767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.512817 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:20.513752 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.514257 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.515259 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.515445 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.515498 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.517794 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:20.517796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:20.517796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:20.517802 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:20.519321 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.519730 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.519731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.519732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.519788 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.520048 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.520049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.520049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.520186 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:20.520239 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.520560 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.520638 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:09:20.520802 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.511947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:45783)
0807 15:09:20.512123 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.512160 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.512668 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.512873 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:20.512876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:20.513472 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:20.513680 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.513685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.514323 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.514324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.515185 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.515188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.515507 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.515507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.515622 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.516119 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:20.516127 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:20.517701 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:20.519396 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:20.519402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:20.519404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:20.519659 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:20.519843 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:20.519843 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:20.519844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:20.519982 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:20.520245 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:20.520252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:20.520413 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 15:09:20.520498 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:09:20.520626 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260807 15:09:20.521260 8869 heartbeater.cc:347] Connected to a master server at 127.18.129.62:37941
I20260807 15:09:20.521373 8869 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:20.521574 8869 heartbeater.cc:511] Master 127.18.129.62:37941 requested a full tablet report, sending...
I20260807 15:09:20.521991 8145 ts_manager.cc:194] Registered new tserver with Master: ab7ba647706144daa018d3b6b345d120 (127.18.129.3:33209)
I20260807 15:09:20.522359 8145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:45783
I20260807 15:09:20.523563 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.516927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53132 (local address 127.18.129.62:37941)
0807 15:09:20.517026 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.517028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.517089 (+ 61us) server_negotiation.cc:407] Connection header received
0807 15:09:20.517156 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.517156 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.517167 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.517220 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:20.517873 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.518528 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.519585 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.519857 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.519920 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.521158 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:20.521160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:20.521160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:20.521167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:20.522557 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.522878 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.522879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.522880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.522937 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:20.523101 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:20.523102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:20.523102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:20.523204 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:20.523367 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.523404 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.523474 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":55,"threads_started":1}
I20260807 15:09:20.524181 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:20.526695 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.524458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.18.129.62:37941)
0807 15:09:20.524492 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.524494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.524597 (+ 103us) server_negotiation.cc:407] Connection header received
0807 15:09:20.524667 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.524668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.524705 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.524758 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:20.524878 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.525322 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.526019 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.526220 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.526273 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.526414 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:20.526514 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:20.526591 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.526608 (+ 17us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.526629 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 15:09:20.526978 8145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:20.528479 8145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:20.530768 8334 tablet_service.cc:1467] Processing CreateTablet for tablet 682af6ff4cad493cb9abe9f3fd195bf1 (DEFAULT_TABLE table=test-table [id=a8cda4e92b06405aa00f3cdb43722cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:20.531003 8334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 682af6ff4cad493cb9abe9f3fd195bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:20.532222 8268 tablet_bootstrap.cc:492] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: Bootstrap starting.
I20260807 15:09:20.532565 8268 tablet_bootstrap.cc:654] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:20.533134 8268 tablet_bootstrap.cc:492] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: No bootstrap required, opened a new log
I20260807 15:09:20.533175 8268 ts_tablet_manager.cc:1397] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:20.533545 8268 raft_consensus.cc:348] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.533648 8268 raft_consensus.cc:374] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:20.533700 8268 raft_consensus.cc:729] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aabdea75bf7b44e59552f47437d71236, State: Initialized, Role: FOLLOWER
I20260807 15:09:20.533744 8268 consensus_queue.cc:260] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.533829 8268 ts_tablet_manager.cc:1428] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:20.534204 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.530228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:33209 (local address 127.0.0.1:44490)
0807 15:09:20.530272 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.530284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.530333 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.530779 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:20.530780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:20.530782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:20.530835 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.530836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.532135 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.532136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.533152 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.533156 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.533935 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.533936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.534010 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.534031 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:09:20.534081 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:09:20.534128 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:09:20.534437 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.530428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.18.129.3:33209)
0807 15:09:20.530606 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.530611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.530621 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:20.530659 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.530661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.530676 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.530756 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:20.530916 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.532016 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.533236 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.533832 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.533931 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.533996 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.534313 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.534359 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260807 15:09:20.534798 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.530228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43015 (local address 127.0.0.1:37676)
0807 15:09:20.530410 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.530419 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.530461 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.530929 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:20.530930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:20.530931 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:20.530963 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.530963 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.532230 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.532231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.533185 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.533188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.534131 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.534131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.534664 (+ 533us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.534669 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:09:20.534715 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:09:20.534748 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":23,"thread_start_us":103,"threads_started":1}
I20260807 15:09:20.534906 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.530434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.18.129.2:43015)
0807 15:09:20.530618 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.530622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.530632 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:20.530668 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.530669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.530681 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.530756 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:20.531033 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.532124 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.533299 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.534018 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.534102 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.534161 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.534754 (+ 593us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.534801 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
I20260807 15:09:20.535120 8761 tablet_service.cc:1467] Processing CreateTablet for tablet 682af6ff4cad493cb9abe9f3fd195bf1 (DEFAULT_TABLE table=test-table [id=a8cda4e92b06405aa00f3cdb43722cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:20.535403 8761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 682af6ff4cad493cb9abe9f3fd195bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:20.538118 8882 tablet_bootstrap.cc:492] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: Bootstrap starting.
I20260807 15:09:20.538702 8882 tablet_bootstrap.cc:654] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:20.538633 8554 tablet_service.cc:1467] Processing CreateTablet for tablet 682af6ff4cad493cb9abe9f3fd195bf1 (DEFAULT_TABLE table=test-table [id=a8cda4e92b06405aa00f3cdb43722cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:20.538952 8554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 682af6ff4cad493cb9abe9f3fd195bf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:20.539222 8882 log.cc:824] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:20.539968 8882 tablet_bootstrap.cc:492] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: No bootstrap required, opened a new log
I20260807 15:09:20.540027 8882 ts_tablet_manager.cc:1397] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:20.541326 8882 raft_consensus.cc:348] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.541460 8882 raft_consensus.cc:374] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:20.541481 8882 raft_consensus.cc:729] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7ba647706144daa018d3b6b345d120, State: Initialized, Role: FOLLOWER
I20260807 15:09:20.541630 8882 consensus_queue.cc:260] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.541652 8885 tablet_bootstrap.cc:492] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: Bootstrap starting.
I20260807 15:09:20.541921 8882 ts_tablet_manager.cc:1428] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:09:20.541939 8869 heartbeater.cc:503] Master 127.18.129.62:37941 was elected leader, sending a full tablet report...
I20260807 15:09:20.542291 8885 tablet_bootstrap.cc:654] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:20.542819 8885 log.cc:824] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:20.543546 8885 tablet_bootstrap.cc:492] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: No bootstrap required, opened a new log
I20260807 15:09:20.543605 8885 ts_tablet_manager.cc:1397] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:09:20.543810 8889 raft_consensus.cc:482] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:20.543880 8889 raft_consensus.cc:504] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.544189 8889 leader_election.cc:288] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5879df268355427f80f25ab29fed940f (127.18.129.2:43015), aabdea75bf7b44e59552f47437d71236 (127.18.129.1:34609)
I20260807 15:09:20.545604 8885 raft_consensus.cc:348] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.545750 8885 raft_consensus.cc:374] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:20.545782 8885 raft_consensus.cc:729] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5879df268355427f80f25ab29fed940f, State: Initialized, Role: FOLLOWER
I20260807 15:09:20.545912 8885 consensus_queue.cc:260] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.546170 8659 heartbeater.cc:503] Master 127.18.129.62:37941 was elected leader, sending a full tablet report...
I20260807 15:09:20.546185 8885 ts_tablet_manager.cc:1428] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:20.547348 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.544316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43015 (local address 127.18.129.3:57923)
0807 15:09:20.544395 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.544407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.544441 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.544628 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:20.544628 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:20.544630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:20.544697 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.544698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.545318 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.545319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.546336 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.546339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.547102 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.547104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.547184 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.547187 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:09:20.547235 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:09:20.547277 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 15:09:20.547612 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.544386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:57923 (local address 127.18.129.2:43015)
0807 15:09:20.544423 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.544425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.544448 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:09:20.544518 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.544519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.544529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.544584 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:20.544766 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.545247 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.546431 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.546989 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.547059 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.547069 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.547512 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.547552 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:09:20.547842 8457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.544394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:38201 (local address 127.18.129.1:34609)
0807 15:09:20.544435 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.544436 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.544579 (+ 143us) server_negotiation.cc:407] Connection header received
0807 15:09:20.544621 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.544622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.544644 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.544699 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:20.544823 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.545536 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.546770 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.547380 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.547445 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.547521 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.547731 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.547775 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:09:20.547853 8595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "682af6ff4cad493cb9abe9f3fd195bf1" candidate_uuid: "ab7ba647706144daa018d3b6b345d120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5879df268355427f80f25ab29fed940f" is_pre_election: true
I20260807 15:09:20.547962 8595 raft_consensus.cc:2454] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab7ba647706144daa018d3b6b345d120 in term 0.
I20260807 15:09:20.548032 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.544339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34609 (local address 127.18.129.3:38201)
0807 15:09:20.544499 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.544509 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.544564 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.544708 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:20.544708 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:20.544710 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:20.544764 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.544765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.545633 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.545634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.546703 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:20.546706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.547556 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.547557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:20.547648 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.547652 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:20.547917 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 15:09:20.547959 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260807 15:09:20.548202 8695 leader_election.cc:302] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5879df268355427f80f25ab29fed940f, ab7ba647706144daa018d3b6b345d120; no voters:
I20260807 15:09:20.548326 8889 raft_consensus.cc:2781] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:20.548379 8889 raft_consensus.cc:482] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:20.548410 8889 raft_consensus.cc:3037] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:20.548367 8377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "682af6ff4cad493cb9abe9f3fd195bf1" candidate_uuid: "ab7ba647706144daa018d3b6b345d120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aabdea75bf7b44e59552f47437d71236" is_pre_election: true
I20260807 15:09:20.548506 8377 raft_consensus.cc:2454] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab7ba647706144daa018d3b6b345d120 in term 0.
I20260807 15:09:20.549088 8889 raft_consensus.cc:504] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.549201 8889 leader_election.cc:288] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [CANDIDATE]: Term 1 election: Requested vote from peers 5879df268355427f80f25ab29fed940f (127.18.129.2:43015), aabdea75bf7b44e59552f47437d71236 (127.18.129.1:34609)
I20260807 15:09:20.549358 8595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "682af6ff4cad493cb9abe9f3fd195bf1" candidate_uuid: "ab7ba647706144daa018d3b6b345d120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5879df268355427f80f25ab29fed940f"
I20260807 15:09:20.549361 8377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "682af6ff4cad493cb9abe9f3fd195bf1" candidate_uuid: "ab7ba647706144daa018d3b6b345d120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aabdea75bf7b44e59552f47437d71236"
I20260807 15:09:20.549430 8595 raft_consensus.cc:3037] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:20.549430 8377 raft_consensus.cc:3037] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:20.550215 8595 raft_consensus.cc:2454] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab7ba647706144daa018d3b6b345d120 in term 1.
I20260807 15:09:20.550215 8377 raft_consensus.cc:2454] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab7ba647706144daa018d3b6b345d120 in term 1.
I20260807 15:09:20.550367 8695 leader_election.cc:302] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5879df268355427f80f25ab29fed940f, ab7ba647706144daa018d3b6b345d120; no voters:
I20260807 15:09:20.550446 8889 raft_consensus.cc:2781] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:20.550572 8889 raft_consensus.cc:686] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 1 LEADER]: Becoming Leader. State: Replica: ab7ba647706144daa018d3b6b345d120, State: Running, Role: LEADER
I20260807 15:09:20.550652 8889 consensus_queue.cc:237] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } }
I20260807 15:09:20.551400 8145 catalog_manager.cc:5495] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab7ba647706144daa018d3b6b345d120 (127.18.129.3). New cstate: current_term: 1 leader_uuid: "ab7ba647706144daa018d3b6b345d120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab7ba647706144daa018d3b6b345d120" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 33209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 } health_report { overall_health: UNKNOWN } } }
I20260807 15:09:20.557011 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.555107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.18.129.3:33209)
0807 15:09:20.555149 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:20.555150 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:20.555161 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:09:20.555229 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:20.555230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:20.555243 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:20.555296 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:20.555416 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.555849 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.556473 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:20.556640 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:20.556681 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:20.556765 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:20.556847 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:20.556899 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:20.556917 (+ 18us) server_negotiation.cc:299] Negotiation successful
0807 15:09:20.556931 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:09:20.558492 8595 raft_consensus.cc:1261] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 1 FOLLOWER]: Refusing update from remote peer ab7ba647706144daa018d3b6b345d120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:20.558493 8377 raft_consensus.cc:1261] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Refusing update from remote peer ab7ba647706144daa018d3b6b345d120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:20.558804 8897 consensus_queue.cc:1035] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aabdea75bf7b44e59552f47437d71236" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:20.558966 8889 consensus_queue.cc:1035] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5879df268355427f80f25ab29fed940f" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:20.561337 8902 mvcc.cc:205] Tried to move back new op lower bound from 7315928516845481984 to 7315928516816113664. Current Snapshot: MvccSnapshot[applied={T|T < 7315928516845481984}]
I20260807 15:09:20.561388 8905 mvcc.cc:205] Tried to move back new op lower bound from 7315928516845481984 to 7315928516816113664. Current Snapshot: MvccSnapshot[applied={T|T < 7315928516845481984}]
I20260807 15:09:20.563215 8145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:53144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:20.563314 8145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:53144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:20.564013 8145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Sending DeleteTablet for 3 replicas of tablet 682af6ff4cad493cb9abe9f3fd195bf1
I20260807 15:09:20.564327 8554 tablet_service.cc:1514] Processing DeleteTablet for tablet 682af6ff4cad493cb9abe9f3fd195bf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:20 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:37676
I20260807 15:09:20.564352 8334 tablet_service.cc:1514] Processing DeleteTablet for tablet 682af6ff4cad493cb9abe9f3fd195bf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:20 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:38814
I20260807 15:09:20.564438 8458 tablet_replica.cc:330] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: stopping tablet replica
I20260807 15:09:20.564489 8458 raft_consensus.cc:2229] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:20.564561 8458 raft_consensus.cc:2258] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:20.564797 8761 tablet_service.cc:1514] Processing DeleteTablet for tablet 682af6ff4cad493cb9abe9f3fd195bf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:20 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:44490
I20260807 15:09:20.564987 8915 tablet_replica.cc:330] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: stopping tablet replica
I20260807 15:09:20.564981 8458 ts_tablet_manager.cc:1905] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:20.565085 8915 raft_consensus.cc:2229] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:20.565178 8915 raft_consensus.cc:2258] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:20.565523 8915 ts_tablet_manager.cc:1905] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:20.566588 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 8205
I20260807 15:09:20.566706 8458 ts_tablet_manager.cc:1918] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:20.566756 8458 log.cc:1196] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/682af6ff4cad493cb9abe9f3fd195bf1
I20260807 15:09:20.566953 8458 ts_tablet_manager.cc:1939] T 682af6ff4cad493cb9abe9f3fd195bf1 P aabdea75bf7b44e59552f47437d71236: Deleting consensus metadata
I20260807 15:09:20.571869 8915 ts_tablet_manager.cc:1918] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:20.572009 8915 log.cc:1196] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/682af6ff4cad493cb9abe9f3fd195bf1
I20260807 15:09:20.572350 8915 ts_tablet_manager.cc:1939] T 682af6ff4cad493cb9abe9f3fd195bf1 P ab7ba647706144daa018d3b6b345d120: Deleting consensus metadata
I20260807 15:09:20.572782 8121 catalog_manager.cc:4841] TS ab7ba647706144daa018d3b6b345d120 (127.18.129.3:33209): tablet 682af6ff4cad493cb9abe9f3fd195bf1 (table test-table [id=a8cda4e92b06405aa00f3cdb43722cf3]) successfully deleted
I20260807 15:09:20.574980 8914 tablet_replica.cc:330] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: stopping tablet replica
I20260807 15:09:20.575105 8914 raft_consensus.cc:2229] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:20.575227 8914 raft_consensus.cc:2258] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:20.575647 8914 ts_tablet_manager.cc:1905] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:09:20.577059 8121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:34609 (error 108)
I20260807 15:09:20.577095 8914 ts_tablet_manager.cc:1918] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:20.577155 8914 log.cc:1196] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/682af6ff4cad493cb9abe9f3fd195bf1
I20260807 15:09:20.577491 8914 ts_tablet_manager.cc:1939] T 682af6ff4cad493cb9abe9f3fd195bf1 P 5879df268355427f80f25ab29fed940f: Deleting consensus metadata
I20260807 15:09:20.577497 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 8454
I20260807 15:09:20.578104 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.577760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34609 (local address 127.0.0.1:38830)
0807 15:09:20.578021 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.578061 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240}
W20260807 15:09:20.579843 8121 catalog_manager.cc:4584] TS aabdea75bf7b44e59552f47437d71236 (127.18.129.1:34609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 682af6ff4cad493cb9abe9f3fd195bf1: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34609: connect: Connection refused (error 111)
I20260807 15:09:20.587054 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 8662
I20260807 15:09:20.587126 8876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.587012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43015 (local address 127.0.0.1:37678)
0807 15:09:20.587068 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.587094 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.2:43015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:09:20.602119 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 8092
I20260807 15:09:20.610546 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37941
--webserver_interface=127.18.129.62
--webserver_port=36019
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:37941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:20.692116 8918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:20.692322 8918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:20.692353 8918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:20.693461 8918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:20.693500 8918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:20.693519 8918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:20.693534 8918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:20.693549 8918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:20.695019 8918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:37941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:37941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=36019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:20.695252 8918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:20.695446 8918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:09:20.700163 8918 server_base.cc:1034] running on GCE node
W20260807 15:09:20.700124 8933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.700171 8931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.700145 8930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.700635 8918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:20.700913 8918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:20.702059 8918 hybrid_clock.cc:648] HybridClock initialized: now 1786115360702042 us; error 29 us; skew 500 ppm
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:20.704394 8918 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:20.704818 8918 webserver.cc:460] Webserver started at http://127.18.129.62:36019/ using document root <none> and password file <none>
I20260807 15:09:20.705013 8918 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:20.705062 8918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:20.706319 8918 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:20.706980 8943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.707129 8918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.707216 8918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "8d42031dc2ab476f9e938ac11e7c49af"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "c69690adaa62b2df7c087020351491b1"
server_key_iv: "805d6540734168d524a97a24f6004d93"
server_key_version: "encryptionkey@0"
I20260807 15:09:20.707540 8918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:20.715277 8918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:20.716908 8918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:20.717031 8918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:20.720904 8918 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:37941
I20260807 15:09:20.720932 9030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:37941 every 8 connection(s)
I20260807 15:09:20.721282 8918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:20.722771 9031 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:09:20.723484 9031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Bootstrap starting.
I20260807 15:09:20.725414 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 8918
I20260807 15:09:20.725739 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:34609
--local_ip_for_outbound_sockets=127.18.129.1
--tserver_master_addrs=127.18.129.62:37941
--webserver_port=33655
--webserver_interface=127.18.129.1
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:20.732933 9031 log.cc:824] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:20.735144 9031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:09:20.735416 9031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af: Bootstrap complete.
I20260807 15:09:20.737555 9031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:20.737771 9031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d42031dc2ab476f9e938ac11e7c49af, State: Initialized, Role: FOLLOWER
I20260807 15:09:20.737911 9031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:20.737967 9031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:20.737995 9031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:20.738041 9031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:09:20.739017 9031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:20.739117 9031 leader_election.cc:302] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d42031dc2ab476f9e938ac11e7c49af; no voters:
I20260807 15:09:20.739274 9031 leader_election.cc:288] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:09:20.739319 9038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:09:20.739430 9038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [term 3 LEADER]: Becoming Leader. State: Replica: 8d42031dc2ab476f9e938ac11e7c49af, State: Running, Role: LEADER
I20260807 15:09:20.739514 9038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } }
I20260807 15:09:20.739540 9031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:20.739825 9041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8d42031dc2ab476f9e938ac11e7c49af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } } }
I20260807 15:09:20.740032 9041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:20.739976 9038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d42031dc2ab476f9e938ac11e7c49af. Latest consensus state: current_term: 3 leader_uuid: "8d42031dc2ab476f9e938ac11e7c49af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d42031dc2ab476f9e938ac11e7c49af" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 37941 } } }
I20260807 15:09:20.740273 9038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d42031dc2ab476f9e938ac11e7c49af [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:20.740417 9056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:20.741271 9056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:20.741374 9056 catalog_manager.cc:1259] Loaded cluster ID: bfcdb60b65c344c7863d6d2bbe9d3d10
I20260807 15:09:20.741403 9056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 15:09:20.741756 9056 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4dc06119ad google::LogMessage::Fail() at ??:0
@ 0x7f4dc061488a google::LogMessage::SendToLog() at ??:0
@ 0x7f4dc06114de google::LogMessage::Flush() at ??:0
@ 0x7f4dc0612a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4dc85320f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4dc853b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4dc24df0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4dc24d4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4dc33386db start_thread at ??:0
@ 0x7f4dbf10f61f clone at ??:0
W20260807 15:09:20.823182 9035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:20.823377 9035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:20.823401 9035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:20.824546 9035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:20.824589 9035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:20.824615 9035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:09:20.826052 9035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:34609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=33655
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:20.826282 9035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:20.826496 9035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:20.831604 9092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.831645 9093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.831691 9035 server_base.cc:1034] running on GCE node
W20260807 15:09:20.831789 9095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.832012 9035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:20.832252 9035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:20.833400 9035 hybrid_clock.cc:648] HybridClock initialized: now 1786115360833385 us; error 32 us; skew 500 ppm
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:20.836055 9035 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:09:20.836489 9035 webserver.cc:460] Webserver started at http://127.18.129.1:33655/ using document root <none> and password file <none>
I20260807 15:09:20.836728 9035 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:20.836794 9035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:20.838091 9035 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:20.838657 9108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.838841 9035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:20.838972 9035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "aabdea75bf7b44e59552f47437d71236"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "172b10d94bf64e373cee7c35fe1c0233"
server_key_iv: "a5b1fae3b9e35afb5fbf3d1de1270b48"
server_key_version: "encryptionkey@0"
I20260807 15:09:20.839426 9035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:20.860447 9035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:20.862895 9035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:20.863052 9035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:20.863329 9035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:20.863709 9035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:20.863760 9035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.863802 9035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:20.863826 9035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.871176 9035 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:34609
I20260807 15:09:20.871207 9297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:34609 every 8 connection(s)
I20260807 15:09:20.871681 9035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:20.881843 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 9035
I20260807 15:09:20.882114 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:43015
--local_ip_for_outbound_sockets=127.18.129.2
--tserver_master_addrs=127.18.129.62:37941
--webserver_port=36109
--webserver_interface=127.18.129.2
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:20.931864 9302 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:20.872455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:58595)
0807 15:09:20.872618 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.872661 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:20.873150 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:20.931262 (+ 58112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
W20260807 15:09:20.932157 9299 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:37941 (0 consecutive failures): Network error: Failed to ping master at 127.18.129.62:37941: Client connection negotiation failed: client connection to 127.18.129.62:37941: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:09:20.932499 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.932387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:45011)
0807 15:09:20.932434 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.932475 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 15:09:20.932847 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:20.932764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:39467)
0807 15:09:20.932806 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:20.932830 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 15:09:20.932955 9299 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:09:20.967612 9303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:20.967826 9303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:20.967860 9303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:20.968986 9303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:20.969028 9303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:20.969072 9303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:09:20.970549 9303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:43015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=36109
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:20.970780 9303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:20.971010 9303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:20.975970 9314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.975994 9317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:20.976070 9315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:20.976230 9303 server_base.cc:1034] running on GCE node
I20260807 15:09:20.976399 9303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:20.976624 9303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:20.977758 9303 hybrid_clock.cc:648] HybridClock initialized: now 1786115360977740 us; error 33 us; skew 500 ppm
Aug 07 15:09:20 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115360, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:20.980057 9303 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:09:20.980491 9303 webserver.cc:460] Webserver started at http://127.18.129.2:36109/ using document root <none> and password file <none>
I20260807 15:09:20.980698 9303 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:20.980747 9303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:20.982016 9303 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:20.982623 9328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.982828 9303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:20.982971 9303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "5879df268355427f80f25ab29fed940f"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "eeb03a8458dded5c88992e8fb11d214f"
server_key_iv: "dd288de80e0bf7b323fec85514aad350"
server_key_version: "encryptionkey@0"
I20260807 15:09:20.983317 9303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:20.996665 9303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:20.998831 9303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:20.999011 9303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:20.999236 9303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:20.999541 9303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:20.999583 9303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:20.999608 9303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:20.999631 9303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:21.005275 9303 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43015
I20260807 15:09:21.005378 9516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43015 every 8 connection(s)
I20260807 15:09:21.005749 9303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:21.006778 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:21.006388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:32823)
0807 15:09:21.006553 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:21.006633 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
W20260807 15:09:21.007185 9518 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:37941 (0 consecutive failures): Network error: Failed to ping master at 127.18.129.62:37941: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
I20260807 15:09:21.007440 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:21.007357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:41841)
0807 15:09:21.007398 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:21.007422 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 15:09:21.007685 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:21.007615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:57595)
0807 15:09:21.007648 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:21.007670 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 15:09:21.007778 9518 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:09:21.007992 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 9303
I20260807 15:09:21.008214 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:33209
--local_ip_for_outbound_sockets=127.18.129.3
--tserver_master_addrs=127.18.129.62:37941
--webserver_port=33927
--webserver_interface=127.18.129.3
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:21.089471 9525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:21.089668 9525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:21.089691 9525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:21.090785 9525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:21.090821 9525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:21.090865 9525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:09:21.092343 9525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:33209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=33927
--tserver_master_addrs=127.18.129.62:37941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:21.092515 9525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:21.092692 9525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:21.097658 9537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:21.097718 9534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:21.097797 9525 server_base.cc:1034] running on GCE node
W20260807 15:09:21.097676 9533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:21.098064 9525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:21.098299 9525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:21.099447 9525 hybrid_clock.cc:648] HybridClock initialized: now 1786115361099430 us; error 33 us; skew 500 ppm
Aug 07 15:09:21 dist-test-slave-5bn2 krb5kdc[7276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:21.101830 9525 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:09:21.102232 9525 webserver.cc:460] Webserver started at http://127.18.129.3:33927/ using document root <none> and password file <none>
I20260807 15:09:21.102427 9525 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:21.102474 9525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:21.103767 9525 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:21.104308 9545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:21.104453 9525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:21.104548 9525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "ab7ba647706144daa018d3b6b345d120"
format_stamp: "Formatted at 2026-08-07 15:09:19 on dist-test-slave-5bn2"
server_key: "86bfe2962014c17ee45236040388afb3"
server_key_iv: "99940323d1f9088b81958a5d34340e4e"
server_key_version: "encryptionkey@0"
I20260807 15:09:21.104874 9525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:21.163139 9525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:21.165140 9525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:21.165290 9525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:21.165516 9525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:21.165830 9525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:21.165874 9525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:21.165905 9525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:21.165925 9525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:21.171591 9525 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:33209
I20260807 15:09:21.171638 9741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:33209 every 8 connection(s)
I20260807 15:09:21.171978 9525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:21.173071 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:21.172692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:54921)
0807 15:09:21.172849 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:21.172927 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
W20260807 15:09:21.173418 9742 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:37941 (0 consecutive failures): Network error: Failed to ping master at 127.18.129.62:37941: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
I20260807 15:09:21.173667 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:21.173588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:47473)
0807 15:09:21.173625 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:21.173650 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 15:09:21.173926 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 9525
I20260807 15:09:21.173940 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:21.173825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:38523)
0807 15:09:21.173891 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:21.173917 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 15:09:21.174050 9742 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:09:21.933570 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:21.933289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:47423)
0807 15:09:21.933445 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:21.933491 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":61,"threads_started":1}
I20260807 15:09:22.008337 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:22.008127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:33619)
0807 15:09:22.008243 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:22.008279 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260807 15:09:22.174544 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:22.174332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:52585)
0807 15:09:22.174453 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:22.174490 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
I20260807 15:09:22.934304 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:22.934044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:60095)
0807 15:09:22.934201 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:22.934248 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":67,"threads_started":1}
I20260807 15:09:23.008971 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:23.008761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:51463)
0807 15:09:23.008881 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:23.008924 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260807 15:09:23.175185 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:23.174969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:51201)
0807 15:09:23.175102 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:23.175141 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260807 15:09:23.935036 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:23.934773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:48131)
0807 15:09:23.934939 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:23.934981 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260807 15:09:24.009610 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:24.009373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:39669)
0807 15:09:24.009507 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:24.009552 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260807 15:09:24.175808 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:24.175609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:39453)
0807 15:09:24.175734 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:24.175766 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260807 15:09:24.935748 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:24.935514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:33395)
0807 15:09:24.935649 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:24.935693 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260807 15:09:25.010281 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:25.010051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:42361)
0807 15:09:25.010184 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:25.010229 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260807 15:09:25.176445 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:25.176198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:57749)
0807 15:09:25.176344 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:25.176393 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260807 15:09:25.936492 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:25.936202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:39681)
0807 15:09:25.936376 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:25.936416 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
I20260807 15:09:26.011029 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:26.010752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:38813)
0807 15:09:26.010933 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:26.010977 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
I20260807 15:09:26.177181 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:26.176935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:53791)
0807 15:09:26.177069 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:26.177120 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260807 15:09:26.937184 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:26.936959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:40901)
0807 15:09:26.937087 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:26.937130 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260807 15:09:27.011737 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.011494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:36245)
0807 15:09:27.011643 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:27.011685 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260807 15:09:27.177829 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.177623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:43049)
0807 15:09:27.177742 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:27.177781 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260807 15:09:27.937873 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.937625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:48211)
0807 15:09:27.937778 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:27.937819 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":63,"threads_started":1}
I20260807 15:09:28.012411 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:28.012172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:36633)
0807 15:09:28.012319 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:28.012359 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260807 15:09:28.178493 9784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:28.178268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:36429)
0807 15:09:28.178410 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:28.178446 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260807 15:09:28.938553 9785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:28.938349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:35439)
0807 15:09:28.938464 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:28.938500 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260807 15:09:29.013034 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:29.012823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:38179)
0807 15:09:29.012942 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:29.012982 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260807 15:09:29.179143 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:29.178935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:54987)
0807 15:09:29.179045 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:29.179096 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260807 15:09:29.939234 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:29.938993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:51073)
0807 15:09:29.939141 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:29.939179 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260807 15:09:30.013684 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.013480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:52193)
0807 15:09:30.013597 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.013631 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260807 15:09:30.179769 9795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.179545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:52491)
0807 15:09:30.179672 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.179712 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260807 15:09:30.940006 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.939773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:43511)
0807 15:09:30.939909 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.939951 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260807 15:09:31.014348 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:31.014111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:45541)
0807 15:09:31.014250 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:31.014294 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260807 15:09:31.180374 9801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:31.180160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:33931)
0807 15:09:31.180280 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:31.180325 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260807 15:09:31.940732 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:31.940493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:37357)
0807 15:09:31.940630 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:31.940675 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260807 15:09:32.015038 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.014803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:54281)
0807 15:09:32.014945 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:32.014985 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260807 15:09:32.181015 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.180789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:46611)
0807 15:09:32.180921 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:32.180964 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260807 15:09:32.941442 9810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.941224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:57057)
0807 15:09:32.941346 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:32.941387 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260807 15:09:33.015714 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.015471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:41945)
0807 15:09:33.015619 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.015662 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260807 15:09:33.181784 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.181545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:58107)
0807 15:09:33.181682 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.181727 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260807 15:09:33.942126 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.941883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:33467)
0807 15:09:33.942020 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.942073 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260807 15:09:34.016387 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:34.016166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:58297)
0807 15:09:34.016292 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:34.016336 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260807 15:09:34.182452 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:34.182241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:50261)
0807 15:09:34.182364 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:34.182401 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260807 15:09:34.942793 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:34.942561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:44093)
0807 15:09:34.942699 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:34.942739 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260807 15:09:35.017117 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:35.016864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:55511)
0807 15:09:35.017003 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:35.017064 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260807 15:09:35.183125 9821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:35.182902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:46573)
0807 15:09:35.183018 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:35.183056 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260807 15:09:35.943570 9824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:35.943347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.1:35851)
0807 15:09:35.943463 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:35.943499 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
I20260807 15:09:36.017863 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.017650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.2:60859)
0807 15:09:36.017775 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.017812 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260807 15:09:36.181504 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 9035
I20260807 15:09:36.187001 9826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.186739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:37941 (local address 127.18.129.3:49527)
0807 15:09:36.186900 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.186940 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":56,"threads_started":1}
I20260807 15:09:36.193411 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 9303
I20260807 15:09:36.202893 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 9525
2026-08-07T15:09:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18342 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:36 dist-test-slave-5bn2 krb5kdc[9830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:36 dist-test-slave-5bn2 krb5kdc[9830](info): set up 2 sockets
Aug 07 15:09:36 dist-test-slave-5bn2 krb5kdc[9830](info): commencing operation
krb5kdc: starting...
W20260807 15:09:37.346468 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:09:37 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:09:37.465698 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45449
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:45449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:37.546766 9852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:37.546998 9852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:37.547026 9852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:37.548156 9852 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:09:37.548197 9852 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:09:37.548213 9852 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:09:37.548228 9852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:37.548244 9852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:37.548262 9852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:37.548285 9852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:37.548306 9852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:37.549769 9852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:45449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:37.550009 9852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:37.550232 9852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:37.555347 9866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:37.555414 9860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:37.555321 9861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:37.555341 9852 server_base.cc:1034] running on GCE node
I20260807 15:09:37.555850 9852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:37.556123 9852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:37.557271 9852 hybrid_clock.cc:648] HybridClock initialized: now 1786115377557252 us; error 32 us; skew 500 ppm
Aug 07 15:09:37 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:37.559710 9852 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:37.560155 9852 webserver.cc:460] Webserver started at http://127.18.129.62:35349/ using document root <none> and password file <none>
I20260807 15:09:37.560348 9852 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:37.560398 9852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:37.560529 9852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:37.561565 9852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "7b6eaf69913b4299bbf7968911821165"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "7229eae38041decccf8539b43b97726e"
server_key_iv: "cb4f639e02ffde552f89e7a8fcdd8a9d"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.561906 9852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "7b6eaf69913b4299bbf7968911821165"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "7229eae38041decccf8539b43b97726e"
server_key_iv: "cb4f639e02ffde552f89e7a8fcdd8a9d"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.563236 9852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:09:37.563937 9882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:37.564091 9852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:37.564182 9852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "7b6eaf69913b4299bbf7968911821165"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "7229eae38041decccf8539b43b97726e"
server_key_iv: "cb4f639e02ffde552f89e7a8fcdd8a9d"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.564244 9852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:37.590893 9852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:37.593192 9852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:37.593320 9852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:37.597335 9852 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:45449
I20260807 15:09:37.597371 9962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:45449 every 8 connection(s)
I20260807 15:09:37.597731 9852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:37.598618 9963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:37.600725 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 9852
I20260807 15:09:37.600821 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:09:37.600986 18948 external_mini_cluster.cc:1428] Setting key 5803c0c9aa6bf4e6e5af139e11bd5844
I20260807 15:09:37.601086 9963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Bootstrap starting.
I20260807 15:09:37.601753 9963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:37.602032 9963 log.cc:824] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:37.602820 9963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: No bootstrap required, opened a new log
I20260807 15:09:37.604995 9963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:09:37.605189 9963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:37.605232 9963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6eaf69913b4299bbf7968911821165, State: Initialized, Role: FOLLOWER
I20260807 15:09:37.605365 9963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:09:37.605432 9963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:37.605453 9963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:37.605499 9963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:37.606120 9963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:09:37.606221 9963 leader_election.cc:302] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b6eaf69913b4299bbf7968911821165; no voters:
I20260807 15:09:37.606381 9963 leader_election.cc:288] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:37.606446 9971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:37.606640 9963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:37.606658 9971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 1 LEADER]: Becoming Leader. State: Replica: 7b6eaf69913b4299bbf7968911821165, State: Running, Role: LEADER
I20260807 15:09:37.606825 9971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:09:37.607242 9973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b6eaf69913b4299bbf7968911821165. Latest consensus state: current_term: 1 leader_uuid: "7b6eaf69913b4299bbf7968911821165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } } }
I20260807 15:09:37.607254 9972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b6eaf69913b4299bbf7968911821165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } } }
I20260807 15:09:37.607328 9973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:37.607350 9972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:37.607597 9980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 15:09:37 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:37.608105 9980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:37.609661 9980 catalog_manager.cc:1347] Generated new cluster ID: 0ed1c2b448e146e299b71f149fe02d63
I20260807 15:09:37.609710 9980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:37.613407 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:37.601497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.18.129.62:45449)
0807 15:09:37.601690 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:37.601698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:37.601723 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:09:37.602175 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:37.602184 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:37.602240 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:37.602390 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:37.603247 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.604133 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.605191 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.605737 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.605816 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:37.608584 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:37.608589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:37.608590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:37.608602 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:37.611757 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:37.612161 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:37.612163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:37.612164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:37.612261 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:37.612453 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:37.612454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:37.612454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:37.612725 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:37.612866 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:37.612979 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 15:09:37.613201 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260807 15:09:37.658460 9980 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:37.659035 9980 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:37.668752 9980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Generated new TSK 0
I20260807 15:09:37.668918 9980 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:09:37.699774 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45449
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:37.781059 9999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:37.781266 9999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:37.781287 9999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:37.782363 9999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:37.782397 9999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:37.782421 9999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:09:37.783838 9999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:37.784052 9999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:37.784273 9999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:37.789260 10007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:37.789289 10006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:37.789278 9999 server_base.cc:1034] running on GCE node
W20260807 15:09:37.789260 10010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:37.789673 9999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:37.789886 9999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:37.791020 9999 hybrid_clock.cc:648] HybridClock initialized: now 1786115377791000 us; error 31 us; skew 500 ppm
Aug 07 15:09:37 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:37.793462 9999 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:09:37.793889 9999 webserver.cc:460] Webserver started at http://127.18.129.1:44275/ using document root <none> and password file <none>
I20260807 15:09:37.794137 9999 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:37.794188 9999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:37.794312 9999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:37.795392 9999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "c8010d4ae6f8495e9e27ece97ed4c7db"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "ee16917db464177d3ee80274f3ab8a52"
server_key_iv: "2d3dfe19b8182e09b50091520cf8a0c6"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.795753 9999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "c8010d4ae6f8495e9e27ece97ed4c7db"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "ee16917db464177d3ee80274f3ab8a52"
server_key_iv: "2d3dfe19b8182e09b50091520cf8a0c6"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.796962 9999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:09:37.797614 10020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:37.797765 9999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:37.797842 9999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "c8010d4ae6f8495e9e27ece97ed4c7db"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "ee16917db464177d3ee80274f3ab8a52"
server_key_iv: "2d3dfe19b8182e09b50091520cf8a0c6"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.797904 9999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:37.816219 9999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:37.817971 9999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:37.818105 9999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:37.818324 9999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:37.818634 9999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:37.818670 9999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:37.818699 9999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:37.818722 9999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:37.824287 9999 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:34113
I20260807 15:09:37.824361 10187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:34113 every 8 connection(s)
I20260807 15:09:37.824661 9999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:09:37 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:37.833256 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:37.825352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:45657 (local address 127.18.129.62:45449)
0807 15:09:37.825432 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:37.825435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:37.825571 (+ 136us) server_negotiation.cc:407] Connection header received
0807 15:09:37.826022 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:37.826023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:37.826033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:37.826086 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:37.827014 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.827519 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.828506 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.828713 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.828775 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:37.830923 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:37.830924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:37.830924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:37.830932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:37.832191 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:37.832532 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:37.832551 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:37.832551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:37.832602 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:37.832779 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:37.832779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:37.832779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:37.832875 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:37.832920 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:37.833118 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 15:09:37.833186 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:09:37.833406 10193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:37.825309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:45657)
0807 15:09:37.825450 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:37.825487 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:37.825948 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:37.826162 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:37.826165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:37.826720 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:37.826939 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:37.826944 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.827602 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.827602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:37.828386 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:37.828389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.828783 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.828784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:37.828878 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:37.829375 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:37.829383 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:37.830799 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:37.832260 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:37.832266 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:37.832268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:37.832476 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:37.832647 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:37.832647 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:37.832648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:37.832737 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:37.832915 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:37.832917 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:37.833025 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 15:09:37.833103 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 15:09:37.833221 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260807 15:09:37.833909 10191 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45449
I20260807 15:09:37.834004 10191 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:37.834184 10191 heartbeater.cc:511] Master 127.18.129.62:45449 requested a full tablet report, sending...
I20260807 15:09:37.834664 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 9999
I20260807 15:09:37.834774 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:37.834765 9908 ts_manager.cc:194] Registered new tserver with Master: c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113)
I20260807 15:09:37.835001 18948 external_mini_cluster.cc:1428] Setting key c43cbb579e4e3d5714c2285ed981a078
I20260807 15:09:37.835552 9908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:45657
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:09:37.859131 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45449
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:37.942548 10196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:37.942750 10196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:37.942781 10196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:37.943974 10196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:37.944018 10196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:37.944053 10196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:09:37.945518 10196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:37.945756 10196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:37.945966 10196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:37.951077 10216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:37.951159 10196 server_base.cc:1034] running on GCE node
W20260807 15:09:37.951085 10203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:37.951259 10211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:37.951568 10196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:37.951819 10196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:37.952968 10196 hybrid_clock.cc:648] HybridClock initialized: now 1786115377952954 us; error 32 us; skew 500 ppm
Aug 07 15:09:37 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:37.955561 10196 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:09:37.956030 10196 webserver.cc:460] Webserver started at http://127.18.129.2:46265/ using document root <none> and password file <none>
I20260807 15:09:37.956269 10196 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:37.956326 10196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:37.956465 10196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:37.957535 10196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "4b80df8a297e46d69cc8c23dcdd1acd6"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "27897afc009ef5444f3b390ac0719dea"
server_key_iv: "0299f4ec68a3c4ba445889787359cd46"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.957897 10196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "4b80df8a297e46d69cc8c23dcdd1acd6"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "27897afc009ef5444f3b390ac0719dea"
server_key_iv: "0299f4ec68a3c4ba445889787359cd46"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.959270 10196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:09:37.959983 10229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:37.960168 10196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:37.960290 10196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "4b80df8a297e46d69cc8c23dcdd1acd6"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "27897afc009ef5444f3b390ac0719dea"
server_key_iv: "0299f4ec68a3c4ba445889787359cd46"
server_key_version: "encryptionkey@0"
I20260807 15:09:37.960369 10196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:37.977025 10196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:37.978785 10196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:37.978946 10196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:37.979158 10196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:37.979460 10196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:37.979496 10196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:37.979524 10196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:37.979557 10196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:37.985036 10196 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43315
I20260807 15:09:37.985077 10403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43315 every 8 connection(s)
I20260807 15:09:37.985380 10196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:09:37 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115377, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:37.993722 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:37.986048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:57557 (local address 127.18.129.62:45449)
0807 15:09:37.986105 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:37.986107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:37.986290 (+ 183us) server_negotiation.cc:407] Connection header received
0807 15:09:37.986775 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:37.986776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:37.986786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:37.986841 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:37.987805 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.988299 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.989244 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.989419 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.989464 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:37.991482 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:37.991484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:37.991484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:37.991490 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:37.992683 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:37.993019 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:37.993020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:37.993021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:37.993066 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:37.993244 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:37.993244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:37.993244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:37.993339 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:37.993382 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:37.993594 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 15:09:37.993654 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:09:37.993875 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:37.986019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:57557)
0807 15:09:37.986180 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:37.986222 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:37.986690 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:37.986945 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:37.986948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:37.987535 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:37.987730 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:37.987736 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.988371 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.988371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:37.989180 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:37.989182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.989473 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.989474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:37.989574 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:37.990020 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:37.990028 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:37.991389 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:37.992748 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:37.992755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:37.992757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:37.992965 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:37.993108 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:37.993109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:37.993109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:37.993203 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:37.993378 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:37.993381 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:37.993487 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 15:09:37.993560 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:09:37.993679 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260807 15:09:37.994019 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 10196
I20260807 15:09:37.994150 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:37.994351 18948 external_mini_cluster.cc:1428] Setting key 0da350d62ab4df6e65111320ea5bb7c0
I20260807 15:09:37.994335 10405 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45449
I20260807 15:09:37.994419 10405 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:37.994601 10405 heartbeater.cc:511] Master 127.18.129.62:45449 requested a full tablet report, sending...
I20260807 15:09:37.994975 9907 ts_manager.cc:194] Registered new tserver with Master: 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315)
I20260807 15:09:37.995347 9907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:57557
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:09:38.019764 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45449
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:38.100665 10411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:38.100891 10411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:38.100914 10411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:38.102048 10411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:38.102090 10411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:38.102116 10411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:09:38.103629 10411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:38.103819 10411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:38.103996 10411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:38.108999 10421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:38.108999 10422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:38.109086 10411 server_base.cc:1034] running on GCE node
W20260807 15:09:38.109007 10426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:38.109414 10411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:38.109619 10411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:38.110751 10411 hybrid_clock.cc:648] HybridClock initialized: now 1786115378110737 us; error 30 us; skew 500 ppm
Aug 07 15:09:38 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115378, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:38.113143 10411 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:09:38.113581 10411 webserver.cc:460] Webserver started at http://127.18.129.3:43055/ using document root <none> and password file <none>
I20260807 15:09:38.113788 10411 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:38.113842 10411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:38.113970 10411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:38.115082 10411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "082e96ba3cf2453590d6a5ae87ac24ff"
format_stamp: "Formatted at 2026-08-07 15:09:38 on dist-test-slave-5bn2"
server_key: "d3037b2e9716e1226ede6bff386509d5"
server_key_iv: "9c4f6d8726498949fe75fa4795f6170e"
server_key_version: "encryptionkey@0"
I20260807 15:09:38.115409 10411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "082e96ba3cf2453590d6a5ae87ac24ff"
format_stamp: "Formatted at 2026-08-07 15:09:38 on dist-test-slave-5bn2"
server_key: "d3037b2e9716e1226ede6bff386509d5"
server_key_iv: "9c4f6d8726498949fe75fa4795f6170e"
server_key_version: "encryptionkey@0"
I20260807 15:09:38.116623 10411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:38.117254 10436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:38.117408 10411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:38.117494 10411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "082e96ba3cf2453590d6a5ae87ac24ff"
format_stamp: "Formatted at 2026-08-07 15:09:38 on dist-test-slave-5bn2"
server_key: "d3037b2e9716e1226ede6bff386509d5"
server_key_iv: "9c4f6d8726498949fe75fa4795f6170e"
server_key_version: "encryptionkey@0"
I20260807 15:09:38.117556 10411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:38.136126 10411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:38.137974 10411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:38.138121 10411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:38.138352 10411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:38.138672 10411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:38.138708 10411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:38.138737 10411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:38.138756 10411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:38.144381 10411 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:36441
I20260807 15:09:38.144436 10602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:36441 every 8 connection(s)
I20260807 15:09:38.144743 10411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:09:38 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115378, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:38.152978 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.145427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:42695 (local address 127.18.129.62:45449)
0807 15:09:38.145478 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.145481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.145691 (+ 210us) server_negotiation.cc:407] Connection header received
0807 15:09:38.146165 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.146166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.146175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.146224 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:38.147174 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.147670 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.148615 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.148786 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.148827 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.150811 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:38.150813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:38.150813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:38.150820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:38.151970 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:38.152314 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:38.152315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:38.152316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:38.152361 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:38.152520 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:38.152521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:38.152521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:38.152613 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:38.152657 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.152857 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.152915 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:09:38.153148 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.145400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:42695)
0807 15:09:38.145575 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:38.145612 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:38.146091 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:38.146278 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:38.146281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:38.146878 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:38.147100 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.147106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.147742 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.147743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.148548 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.148550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.148841 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.148842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.148951 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.149368 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:38.149376 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:38.150722 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:38.152039 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:38.152046 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:38.152047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:38.152255 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:38.152400 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:38.152401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:38.152401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:38.152484 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:38.152655 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:38.152658 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:38.152766 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 15:09:38.152843 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:09:38.152958 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260807 15:09:38.153582 10603 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45449
I20260807 15:09:38.153663 10603 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:38.153802 10603 heartbeater.cc:511] Master 127.18.129.62:45449 requested a full tablet report, sending...
I20260807 15:09:38.154150 9907 ts_manager.cc:194] Registered new tserver with Master: 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441)
I20260807 15:09:38.154511 9907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:42695
I20260807 15:09:38.154798 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 10411
I20260807 15:09:38.154907 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:38.155076 18948 external_mini_cluster.cc:1428] Setting key f9295104bd3ccb0844f441d5124f23ff
I20260807 15:09:38.155807 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:38.162489 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.157043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.18.129.62:45449)
0807 15:09:38.157100 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.157102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.157192 (+ 90us) server_negotiation.cc:407] Connection header received
0807 15:09:38.157253 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.157254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.157262 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.157303 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:38.157814 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.158275 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.159046 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.159256 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.159311 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.160378 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:38.160380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:38.160380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:38.160387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:38.161410 (+ 1023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:38.161716 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:38.161716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:38.161717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:38.161771 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:38.161931 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:38.161932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:38.161932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:38.162075 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:38.162167 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.162269 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.162346 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:09:38.163977 9907 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:38.165585 9907 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:38.171801 10619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.167753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43315 (local address 127.0.0.1:58256)
0807 15:09:38.168148 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:38.168157 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:38.168203 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:38.168429 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:38.168430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:38.168432 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:38.168479 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.168481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.169731 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.169732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.170829 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.170831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.171507 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.171508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.171585 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.171599 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:09:38.171664 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:09:38.171721 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":28,"thread_start_us":44,"threads_started":1}
I20260807 15:09:38.171916 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.167624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45620 (local address 127.18.129.3:36441)
0807 15:09:38.167793 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.167799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.168091 (+ 292us) server_negotiation.cc:407] Connection header received
0807 15:09:38.168138 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.168140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.168156 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.168243 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:38.168471 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.169597 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.170799 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.171567 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.171649 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.171709 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.171795 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.171835 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260807 15:09:38.172153 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.167735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.18.129.1:34113)
0807 15:09:38.167920 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.167924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.168139 (+ 215us) server_negotiation.cc:407] Connection header received
0807 15:09:38.168187 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.168190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.168206 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.168279 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:38.168535 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.169336 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.170883 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.171599 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.171655 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.171710 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.172043 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.172092 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
I20260807 15:09:38.172216 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.167946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.18.129.2:43315)
0807 15:09:38.168108 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.168112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.168210 (+ 98us) server_negotiation.cc:407] Connection header received
0807 15:09:38.168257 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.168259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.168273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.168345 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:38.168580 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.169641 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.170930 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.171427 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.171491 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.171549 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.172109 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.172149 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260807 15:09:38.171895 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.167534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:36441 (local address 127.0.0.1:45620)
0807 15:09:38.167948 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:38.167988 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:38.168059 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:38.168318 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:38.168319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:38.168322 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:38.168393 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.168395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.169728 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.169729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.170725 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.170728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.171657 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.171657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.171731 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.171735 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:38.171785 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:09:38.171816 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":40,"thread_start_us":76,"threads_started":1}
I20260807 15:09:38.171976 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.167534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34113 (local address 127.0.0.1:60628)
0807 15:09:38.167948 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:38.167988 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:38.168059 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:38.168279 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:38.168282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:38.168284 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:38.168452 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.168456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.169436 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.169437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.170785 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.170787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.171746 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.171747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.171815 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.171820 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:09:38.171868 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:09:38.171910 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260807 15:09:38.173034 10301 tablet_service.cc:1467] Processing CreateTablet for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 (DEFAULT_TABLE table=test-table [id=6a522e0cfd934c02b07550bb12763b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:38.173035 10505 tablet_service.cc:1467] Processing CreateTablet for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 (DEFAULT_TABLE table=test-table [id=6a522e0cfd934c02b07550bb12763b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:38.173406 10301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:38.173405 10505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:38.175230 10099 tablet_service.cc:1467] Processing CreateTablet for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 (DEFAULT_TABLE table=test-table [id=6a522e0cfd934c02b07550bb12763b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:38.175544 10099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:38.176801 10624 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: Bootstrap starting.
I20260807 15:09:38.177155 10625 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Bootstrap starting.
I20260807 15:09:38.177503 10624 tablet_bootstrap.cc:654] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:38.177781 10625 tablet_bootstrap.cc:654] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:38.178084 10624 log.cc:824] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:38.178265 10625 log.cc:824] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:38.178423 10627 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Bootstrap starting.
I20260807 15:09:38.178963 10624 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: No bootstrap required, opened a new log
I20260807 15:09:38.179040 10624 ts_tablet_manager.cc:1397] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:09:38.179138 10627 tablet_bootstrap.cc:654] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:38.179148 10625 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: No bootstrap required, opened a new log
I20260807 15:09:38.179201 10625 ts_tablet_manager.cc:1397] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:38.179613 10627 log.cc:824] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:38.180335 10627 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: No bootstrap required, opened a new log
I20260807 15:09:38.180397 10627 ts_tablet_manager.cc:1397] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:09:38.180950 10624 raft_consensus.cc:348] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.181143 10624 raft_consensus.cc:374] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:38.181197 10624 raft_consensus.cc:729] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082e96ba3cf2453590d6a5ae87ac24ff, State: Initialized, Role: FOLLOWER
I20260807 15:09:38.181193 10625 raft_consensus.cc:348] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.181346 10625 raft_consensus.cc:374] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:38.181317 10624 consensus_queue.cc:260] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.181378 10625 raft_consensus.cc:729] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b80df8a297e46d69cc8c23dcdd1acd6, State: Initialized, Role: FOLLOWER
I20260807 15:09:38.181479 10625 consensus_queue.cc:260] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.181617 10624 ts_tablet_manager.cc:1428] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:38.181727 10625 ts_tablet_manager.cc:1428] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:09:38.181782 10405 heartbeater.cc:503] Master 127.18.129.62:45449 was elected leader, sending a full tablet report...
I20260807 15:09:38.181969 10627 raft_consensus.cc:348] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.182147 10627 raft_consensus.cc:374] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:38.182199 10603 heartbeater.cc:503] Master 127.18.129.62:45449 was elected leader, sending a full tablet report...
I20260807 15:09:38.182209 10627 raft_consensus.cc:729] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8010d4ae6f8495e9e27ece97ed4c7db, State: Initialized, Role: FOLLOWER
I20260807 15:09:38.182322 10627 consensus_queue.cc:260] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.182595 10627 ts_tablet_manager.cc:1428] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:09:38.182658 10191 heartbeater.cc:503] Master 127.18.129.62:45449 was elected leader, sending a full tablet report...
W20260807 15:09:38.235955 10406 tablet.cc:2369] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:38.240813 10635 raft_consensus.cc:482] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:38.240950 10635 raft_consensus.cc:504] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.241289 10635 leader_election.cc:288] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441), c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113)
I20260807 15:09:38.244971 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.241570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:46695 (local address 127.18.129.3:36441)
0807 15:09:38.241620 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.241622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.241902 (+ 280us) server_negotiation.cc:407] Connection header received
0807 15:09:38.241942 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.241943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.241953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.242003 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:38.242141 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.242795 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.244059 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.244643 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.244719 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.244729 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.244858 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.244894 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:09:38.244972 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.241475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:36441 (local address 127.18.129.2:46695)
0807 15:09:38.241674 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:38.241682 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:38.241726 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:38.242012 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:38.242013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:38.242014 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:38.242080 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.242081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.242888 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.242889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.243983 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.243985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.244719 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.244719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.244778 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.244782 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:38.244837 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:09:38.244879 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":44,"thread_start_us":119,"threads_started":1}
I20260807 15:09:38.245060 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.241566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:44915 (local address 127.18.129.1:34113)
0807 15:09:38.241620 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.241622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.241631 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:09:38.241661 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.241662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.241672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.241742 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:38.241906 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.242651 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.244185 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.244753 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.244814 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.244822 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.244960 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.245001 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:09:38.245081 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.241475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34113 (local address 127.18.129.2:44915)
0807 15:09:38.241520 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:38.241542 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:38.241586 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:38.241760 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:38.241762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:38.241765 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:38.241833 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.241835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.242721 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.242722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.243963 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:38.243967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.244819 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.244820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:38.244893 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.244896 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:09:38.244945 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:09:38.244978 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":45}
I20260807 15:09:38.245352 10531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28cb3e1190f44c9ab5208a61b3c1cbe3" candidate_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" is_pre_election: true
I20260807 15:09:38.245384 10124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28cb3e1190f44c9ab5208a61b3c1cbe3" candidate_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" is_pre_election: true
I20260807 15:09:38.245523 10531 raft_consensus.cc:2454] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b80df8a297e46d69cc8c23dcdd1acd6 in term 0.
I20260807 15:09:38.245522 10124 raft_consensus.cc:2454] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b80df8a297e46d69cc8c23dcdd1acd6 in term 0.
I20260807 15:09:38.245754 10231 leader_election.cc:302] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 082e96ba3cf2453590d6a5ae87ac24ff, 4b80df8a297e46d69cc8c23dcdd1acd6; no voters:
I20260807 15:09:38.245900 10635 raft_consensus.cc:2781] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:38.245945 10635 raft_consensus.cc:482] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:38.245968 10635 raft_consensus.cc:3037] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:38.246747 10635 raft_consensus.cc:504] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.246908 10635 leader_election.cc:288] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [CANDIDATE]: Term 1 election: Requested vote from peers 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441), c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113)
I20260807 15:09:38.247067 10124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28cb3e1190f44c9ab5208a61b3c1cbe3" candidate_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db"
I20260807 15:09:38.247069 10531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28cb3e1190f44c9ab5208a61b3c1cbe3" candidate_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "082e96ba3cf2453590d6a5ae87ac24ff"
I20260807 15:09:38.247150 10531 raft_consensus.cc:3037] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:38.247151 10124 raft_consensus.cc:3037] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:38.248026 10531 raft_consensus.cc:2454] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b80df8a297e46d69cc8c23dcdd1acd6 in term 1.
I20260807 15:09:38.248026 10124 raft_consensus.cc:2454] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b80df8a297e46d69cc8c23dcdd1acd6 in term 1.
I20260807 15:09:38.248202 10236 leader_election.cc:302] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b80df8a297e46d69cc8c23dcdd1acd6, c8010d4ae6f8495e9e27ece97ed4c7db; no voters:
I20260807 15:09:38.248301 10635 raft_consensus.cc:2781] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:38.248456 10635 raft_consensus.cc:686] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 LEADER]: Becoming Leader. State: Replica: 4b80df8a297e46d69cc8c23dcdd1acd6, State: Running, Role: LEADER
I20260807 15:09:38.248543 10635 consensus_queue.cc:237] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:38.249426 9908 catalog_manager.cc:5495] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2). New cstate: current_term: 1 leader_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } health_report { overall_health: UNKNOWN } } }
I20260807 15:09:38.266557 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.264521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.18.129.2:43315)
0807 15:09:38.264578 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.264580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.264587 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:09:38.264619 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.264620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.264636 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.264693 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:38.264798 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.265258 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.265941 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.266129 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.266167 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.266278 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:38.266385 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:38.266450 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.266463 (+ 13us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.266481 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:09:38.268231 10124 raft_consensus.cc:1261] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Refusing update from remote peer 4b80df8a297e46d69cc8c23dcdd1acd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:38.268251 10531 raft_consensus.cc:1261] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 1 FOLLOWER]: Refusing update from remote peer 4b80df8a297e46d69cc8c23dcdd1acd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:38.268563 10635 consensus_queue.cc:1035] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:38.268705 10638 consensus_queue.cc:1035] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:38.270839 10643 mvcc.cc:205] Tried to move back new op lower bound from 7315928589384220672 to 7315928589306753024. Current Snapshot: MvccSnapshot[applied={T|T < 7315928589384220672}]
I20260807 15:09:38.271344 10646 mvcc.cc:205] Tried to move back new op lower bound from 7315928589384220672 to 7315928589306753024. Current Snapshot: MvccSnapshot[applied={T|T < 7315928589384220672}]
I20260807 15:09:38.271814 10644 mvcc.cc:205] Tried to move back new op lower bound from 7315928589384220672 to 7315928589306753024. Current Snapshot: MvccSnapshot[applied={T|T < 7315928589384220672}]
I20260807 15:09:38.273870 9908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:38.273957 9908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:38.274681 9908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Sending DeleteTablet for 3 replicas of tablet 28cb3e1190f44c9ab5208a61b3c1cbe3
I20260807 15:09:38.274993 10505 tablet_service.cc:1514] Processing DeleteTablet for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:38 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:45620
I20260807 15:09:38.275019 10301 tablet_service.cc:1514] Processing DeleteTablet for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:38 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:58256
I20260807 15:09:38.275216 10659 tablet_replica.cc:330] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: stopping tablet replica
I20260807 15:09:38.275220 10099 tablet_service.cc:1514] Processing DeleteTablet for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:38 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:60628
I20260807 15:09:38.275341 10659 raft_consensus.cc:2229] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:38.275403 10664 tablet_replica.cc:330] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: stopping tablet replica
I20260807 15:09:38.275473 10659 raft_consensus.cc:2258] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:38.275485 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 9999
I20260807 15:09:38.275547 10664 raft_consensus.cc:2229] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:38.275647 10664 raft_consensus.cc:2258] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:38.275638 10658 tablet_replica.cc:330] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: stopping tablet replica
I20260807 15:09:38.275722 10658 raft_consensus.cc:2229] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:38.275842 10658 raft_consensus.cc:2258] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:38.280277 10658 ts_tablet_manager.cc:1905] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:38.281999 10658 ts_tablet_manager.cc:1918] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:38.282090 10658 log.cc:1196] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/28cb3e1190f44c9ab5208a61b3c1cbe3
I20260807 15:09:38.282377 10658 ts_tablet_manager.cc:1939] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 082e96ba3cf2453590d6a5ae87ac24ff: Deleting consensus metadata
I20260807 15:09:38.282902 9883 catalog_manager.cc:4841] TS 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441): tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 (table test-table [id=6a522e0cfd934c02b07550bb12763b2e]) successfully deleted
I20260807 15:09:38.283787 10659 ts_tablet_manager.cc:1905] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:38.285395 10659 ts_tablet_manager.cc:1918] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:38.285481 10659 log.cc:1196] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/28cb3e1190f44c9ab5208a61b3c1cbe3
I20260807 15:09:38.285845 10659 ts_tablet_manager.cc:1939] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P 4b80df8a297e46d69cc8c23dcdd1acd6: Deleting consensus metadata
I20260807 15:09:38.287127 9883 catalog_manager.cc:4841] TS 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315): tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 (table test-table [id=6a522e0cfd934c02b07550bb12763b2e]) successfully deleted
W20260807 15:09:38.292560 9888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:34113 (error 108)
I20260807 15:09:38.292802 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 10196
I20260807 15:09:38.295908 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.293121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34113 (local address 127.0.0.1:60630)
0807 15:09:38.295802 (+ 2681us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:38.295867 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2651}
W20260807 15:09:38.296128 9888 catalog_manager.cc:4584] TS c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34113: connect: Connection refused (error 111)
I20260807 15:09:38.303117 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 10411
I20260807 15:09:38.316435 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 9852
I20260807 15:09:58.327109 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45449
--webserver_interface=127.18.129.62
--webserver_port=35349
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:45449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:58.412853 10668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:58.413066 10668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:58.413105 10668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:58.414250 10668 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:09:58.414290 10668 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:09:58.414307 10668 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:09:58.414321 10668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:58.414337 10668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:58.414353 10668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:58.414376 10668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:58.414400 10668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:58.415897 10668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:45449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:58.416157 10668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:58.416368 10668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:58.425818 10678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.425901 10668 server_base.cc:1034] running on GCE node
W20260807 15:09:58.425818 10681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.425827 10677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.426362 10668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:58.426663 10668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:58.427800 10668 hybrid_clock.cc:648] HybridClock initialized: now 1786115398427777 us; error 37 us; skew 500 ppm
Aug 07 15:09:58 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:58.433461 10668 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:09:58.433907 10668 webserver.cc:460] Webserver started at http://127.18.129.62:35349/ using document root <none> and password file <none>
I20260807 15:09:58.434126 10668 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:58.434185 10668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:58.435618 10668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:09:58.436362 10690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.436553 10668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:58.436645 10668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "7b6eaf69913b4299bbf7968911821165"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "7229eae38041decccf8539b43b97726e"
server_key_iv: "cb4f639e02ffde552f89e7a8fcdd8a9d"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.437079 10668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:58.445788 10668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:58.447710 10668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:58.447820 10668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:58.452044 10668 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:45449
I20260807 15:09:58.452086 10779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:45449 every 8 connection(s)
I20260807 15:09:58.452491 10668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:58.452886 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 10668
I20260807 15:09:58.453270 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:34113
--local_ip_for_outbound_sockets=127.18.129.1
--tserver_master_addrs=127.18.129.62:45449
--webserver_port=44275
--webserver_interface=127.18.129.1
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:58.454416 10785 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:09:58.455289 10785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Bootstrap starting.
I20260807 15:09:58.464460 10785 log.cc:824] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:58.466504 10785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:09:58.466810 10785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Bootstrap complete.
I20260807 15:09:58.469059 10785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:09:58.469292 10785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6eaf69913b4299bbf7968911821165, State: Initialized, Role: FOLLOWER
I20260807 15:09:58.469440 10785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:09:58.469506 10785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:58.469542 10785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:58.469597 10785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:09:58.470634 10785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:09:58.470721 10785 leader_election.cc:302] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b6eaf69913b4299bbf7968911821165; no voters:
I20260807 15:09:58.470913 10785 leader_election.cc:288] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:09:58.470959 10789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:09:58.471138 10789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 2 LEADER]: Becoming Leader. State: Replica: 7b6eaf69913b4299bbf7968911821165, State: Running, Role: LEADER
I20260807 15:09:58.471185 10785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:58.471293 10789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:09:58.471640 10791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b6eaf69913b4299bbf7968911821165. Latest consensus state: current_term: 2 leader_uuid: "7b6eaf69913b4299bbf7968911821165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } } }
I20260807 15:09:58.471603 10794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7b6eaf69913b4299bbf7968911821165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } } }
I20260807 15:09:58.471774 10794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:58.471720 10791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:58.472280 10801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:58.473268 10801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:58.473373 10801 catalog_manager.cc:1259] Loaded cluster ID: 0ed1c2b448e146e299b71f149fe02d63
I20260807 15:09:58.473403 10801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:58.474560 10801 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:58.474748 10801 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Loaded TSK: 0
I20260807 15:09:58.496541 10801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Generated new TSK 1
I20260807 15:09:58.496661 10801 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:09:58.538379 10787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:58.538589 10787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:58.538620 10787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:58.540050 10787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:58.540098 10787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:58.540134 10787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:09:58.541656 10787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:34113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=44275
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:58.541884 10787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:58.542101 10787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:58.548229 10831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.548286 10787 server_base.cc:1034] running on GCE node
W20260807 15:09:58.548204 10824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.548437 10820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.548698 10787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:58.548930 10787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:58.550148 10787 hybrid_clock.cc:648] HybridClock initialized: now 1786115398550120 us; error 43 us; skew 500 ppm
Aug 07 15:09:58 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:58.552683 10787 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:09:58.553119 10787 webserver.cc:460] Webserver started at http://127.18.129.1:44275/ using document root <none> and password file <none>
I20260807 15:09:58.553328 10787 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:58.553386 10787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:58.554701 10787 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:58.555416 10840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.555622 10787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:09:58.555708 10787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "c8010d4ae6f8495e9e27ece97ed4c7db"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "ee16917db464177d3ee80274f3ab8a52"
server_key_iv: "2d3dfe19b8182e09b50091520cf8a0c6"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.556058 10787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:58.566005 10787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:58.568167 10787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:58.568310 10787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:58.568544 10787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:58.569061 10849 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:09:58.571187 10787 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:09:58.571238 10787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:09:58.571273 10787 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:09:58.572014 10787 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:09:58.572050 10787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:09:58.572139 10849 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Bootstrap starting.
I20260807 15:09:58.578795 10787 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:34113
I20260807 15:09:58.579241 10787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:58.579593 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 10787
I20260807 15:09:58.579874 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:43315
--local_ip_for_outbound_sockets=127.18.129.2
--tserver_master_addrs=127.18.129.62:45449
--webserver_port=46265
--webserver_interface=127.18.129.2
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:58.579615 11017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:34113 every 8 connection(s)
I20260807 15:09:58.583774 10849 log.cc:824] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:58.587020 10849 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:09:58.587422 10849 tablet_bootstrap.cc:492] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Bootstrap complete.
I20260807 15:09:58.588666 10849 ts_tablet_manager.cc:1397] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260807 15:09:58.590142 10849 raft_consensus.cc:348] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:58.590596 10849 raft_consensus.cc:729] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8010d4ae6f8495e9e27ece97ed4c7db, State: Initialized, Role: FOLLOWER
I20260807 15:09:58.590817 10849 consensus_queue.cc:260] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:58.591210 10849 ts_tablet_manager.cc:1428] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Aug 07 15:09:58 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:58.600159 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.584765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:46991)
0807 15:09:58.586997 (+ 2232us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:58.587052 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:58.587678 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:58.588405 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:58.588410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:58.588779 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:58.589081 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.589089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.590957 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.590958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.592229 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.592232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.593056 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.593057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.593216 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.593863 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:58.593873 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:58.595711 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:58.598287 (+ 2576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:58.598295 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:58.598298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:58.598605 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:58.598912 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:58.598913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:58.598913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:58.599046 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:58.599497 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:58.599504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:58.599663 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 15:09:58.599777 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:09:58.599946 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2179,"thread_start_us":68,"threads_started":1}
I20260807 15:09:58.600199 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.585111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:46991 (local address 127.18.129.62:45449)
0807 15:09:58.587934 (+ 2823us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.587944 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.587985 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:09:58.588123 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.588129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.588188 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.589225 (+ 1037us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:58.589329 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.590794 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.592340 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.592940 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.593037 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.595835 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:58.595839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:58.595840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:58.595853 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:58.598188 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.598690 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.598692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.598693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.598808 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.599113 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.599114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.599114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.599396 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:58.599473 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.599840 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.599973 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2766,"thread_start_us":91,"threads_started":1}
I20260807 15:09:58.601076 11018 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45449
I20260807 15:09:58.601186 11018 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:58.601361 11018 heartbeater.cc:511] Master 127.18.129.62:45449 requested a full tablet report, sending...
I20260807 15:09:58.602142 10723 ts_manager.cc:194] Registered new tserver with Master: c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113)
I20260807 15:09:58.603341 10723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:46991
I20260807 15:09:58.604312 11018 heartbeater.cc:503] Master 127.18.129.62:45449 was elected leader, sending a full tablet report...
I20260807 15:09:58.609067 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.603082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41444 (local address 127.18.129.1:34113)
0807 15:09:58.605013 (+ 1931us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.605017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.605026 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:09:58.605059 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.605061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.605077 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.605167 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:58.605380 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.606402 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.607941 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.608647 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.608732 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.608786 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.608959 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.609000 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1898,"thread_start_us":75,"threads_started":1}
I20260807 15:09:58.609066 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.602771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34113 (local address 127.0.0.1:41444)
0807 15:09:58.603710 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:58.603741 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:58.604866 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:58.605193 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:58.605196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:58.605199 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:58.605296 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.605298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.606488 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.606489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.607829 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.607833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.608744 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.608745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.608847 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.608861 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:09:58.608933 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:09:58.608987 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"thread_start_us":148,"threads_started":1}
I20260807 15:09:58.609848 10911 tablet_service.cc:1514] Processing DeleteTablet for tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:38 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41444
I20260807 15:09:58.610481 11032 tablet_replica.cc:330] stopping tablet replica
I20260807 15:09:58.610591 11032 raft_consensus.cc:2229] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:58.610659 11032 raft_consensus.cc:2258] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:58.611007 11032 ts_tablet_manager.cc:1905] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:58.613662 11032 ts_tablet_manager.cc:1918] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:58.613741 11032 log.cc:1196] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal/wals/28cb3e1190f44c9ab5208a61b3c1cbe3
I20260807 15:09:58.613978 11032 ts_tablet_manager.cc:1939] T 28cb3e1190f44c9ab5208a61b3c1cbe3 P c8010d4ae6f8495e9e27ece97ed4c7db: Deleting consensus metadata
I20260807 15:09:58.614554 10695 catalog_manager.cc:4841] TS c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113): tablet 28cb3e1190f44c9ab5208a61b3c1cbe3 (table test-table [id=6a522e0cfd934c02b07550bb12763b2e]) successfully deleted
W20260807 15:09:58.669965 11022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:58.670198 11022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:58.670234 11022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:58.671470 11022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:58.671517 11022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:58.671554 11022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:09:58.673115 11022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:43315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=46265
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:58.673352 11022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:58.673573 11022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:58.678591 11038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.678615 11039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.678596 11042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.678794 11022 server_base.cc:1034] running on GCE node
I20260807 15:09:58.679041 11022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:58.679275 11022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:58.680413 11022 hybrid_clock.cc:648] HybridClock initialized: now 1786115398680432 us; error 62 us; skew 500 ppm
Aug 07 15:09:58 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:58.682987 11022 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:09:58.683394 11022 webserver.cc:460] Webserver started at http://127.18.129.2:46265/ using document root <none> and password file <none>
I20260807 15:09:58.683625 11022 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:58.683691 11022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:58.685024 11022 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:58.685652 11053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.685813 11022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:58.685904 11022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "4b80df8a297e46d69cc8c23dcdd1acd6"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "27897afc009ef5444f3b390ac0719dea"
server_key_iv: "0299f4ec68a3c4ba445889787359cd46"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.686264 11022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:58.707512 11022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:58.709693 11022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:58.709839 11022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:58.710093 11022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:58.710429 11022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:58.710464 11022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.710495 11022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:58.710520 11022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.716157 11022 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43315
I20260807 15:09:58.716226 11235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43315 every 8 connection(s)
I20260807 15:09:58.716557 11022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:09:58 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:58.725658 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.717367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:49625 (local address 127.18.129.62:45449)
0807 15:09:58.717432 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.717435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.717696 (+ 261us) server_negotiation.cc:407] Connection header received
0807 15:09:58.718165 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.718166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.718180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.718246 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:58.719263 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.719786 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.720922 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.721135 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.721193 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.723406 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:58.723407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:58.723407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:58.723415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:58.724496 (+ 1081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.724847 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.724848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.724848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.724893 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.725105 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.725106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.725106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.725201 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:58.725248 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.725517 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.725586 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:09:58.725831 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.717361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:49625)
0807 15:09:58.717525 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:58.717569 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:58.718054 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:58.718302 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:58.718306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:58.718948 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:58.719157 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.719163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.719862 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.719862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.720822 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.720824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.721217 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.721218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.721333 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.721868 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:58.721876 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:58.723299 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:58.724570 (+ 1271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:58.724577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:58.724578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:58.724790 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:58.724938 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:58.724939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:58.724939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:58.725028 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:58.725277 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:58.725284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:58.725411 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 15:09:58.725502 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:09:58.725630 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260807 15:09:58.726013 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 11022
I20260807 15:09:58.726312 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:36441
--local_ip_for_outbound_sockets=127.18.129.3
--tserver_master_addrs=127.18.129.62:45449
--webserver_port=43055
--webserver_interface=127.18.129.3
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:09:58.726380 11237 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45449
I20260807 15:09:58.726472 11237 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:58.726679 11237 heartbeater.cc:511] Master 127.18.129.62:45449 requested a full tablet report, sending...
I20260807 15:09:58.727111 10723 ts_manager.cc:194] Registered new tserver with Master: 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315)
I20260807 15:09:58.727483 10723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:49625
W20260807 15:09:58.809641 11246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:58.809849 11246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:58.809872 11246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:58.811021 11246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:58.811058 11246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:58.811084 11246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:09:58.812484 11246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:36441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=43055
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:09:58.812677 11246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:58.812857 11246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:58.817905 11255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.817999 11246 server_base.cc:1034] running on GCE node
W20260807 15:09:58.817965 11258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.817907 11256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.818406 11246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:58.818638 11246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:58.819803 11246 hybrid_clock.cc:648] HybridClock initialized: now 1786115398819787 us; error 29 us; skew 500 ppm
Aug 07 15:09:58 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:58.822366 11246 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:09:58.822841 11246 webserver.cc:460] Webserver started at http://127.18.129.3:43055/ using document root <none> and password file <none>
I20260807 15:09:58.823097 11246 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:58.823160 11246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:58.824483 11246 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:58.825006 11267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.825160 11246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:58.825242 11246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "082e96ba3cf2453590d6a5ae87ac24ff"
format_stamp: "Formatted at 2026-08-07 15:09:38 on dist-test-slave-5bn2"
server_key: "d3037b2e9716e1226ede6bff386509d5"
server_key_iv: "9c4f6d8726498949fe75fa4795f6170e"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.825618 11246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:58.833431 11246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:58.835323 11246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:58.835464 11246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:58.835669 11246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:58.835959 11246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:58.836004 11246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.836037 11246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:58.836062 11246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.842001 11246 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:36441
I20260807 15:09:58.842041 11447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:36441 every 8 connection(s)
I20260807 15:09:58.842358 11246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:09:58 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:09:58.851117 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.843072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:56255 (local address 127.18.129.62:45449)
0807 15:09:58.843139 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.843141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.843343 (+ 202us) server_negotiation.cc:407] Connection header received
0807 15:09:58.843893 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.843894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.843905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.843952 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:58.844824 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.845319 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.846322 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.846516 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.846565 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.848619 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:58.848620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:58.848620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:58.848627 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:58.849834 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.850212 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.850213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.850213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.850260 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.850504 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.850504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.850504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.850598 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:58.850643 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.850972 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.851038 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:09:58.851292 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.843062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:56255)
0807 15:09:58.843217 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:58.843253 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:58.843800 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:58.844014 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:58.844017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:58.844566 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:58.844757 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.844761 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.845383 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.845383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.846241 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.846243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.846586 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.846587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.846693 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.847138 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:58.847146 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:58.848536 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:58.849909 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:58.849916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:58.849917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:58.850152 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:58.850308 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:58.850308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:58.850308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:58.850456 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:58.850685 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:58.850692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:58.850813 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 15:09:58.850955 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 15:09:58.851086 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260807 15:09:58.851769 11448 heartbeater.cc:347] Connected to a master server at 127.18.129.62:45449
I20260807 15:09:58.851877 11448 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:58.851953 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 11246
I20260807 15:09:58.852073 11448 heartbeater.cc:511] Master 127.18.129.62:45449 requested a full tablet report, sending...
I20260807 15:09:58.852479 10723 ts_manager.cc:194] Registered new tserver with Master: 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441)
I20260807 15:09:58.852921 10723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:56255
I20260807 15:09:58.858166 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.852340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.18.129.62:45449)
0807 15:09:58.852384 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.852385 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.852650 (+ 265us) server_negotiation.cc:407] Connection header received
0807 15:09:58.852696 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.852697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.852707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.852780 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:58.853403 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.853850 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.854572 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.854777 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.854830 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.855927 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:58.855928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:58.855929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:58.855935 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:58.857114 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.857465 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.857466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.857466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.857520 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.857691 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.857692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.857692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.857793 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:58.857972 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.858009 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.858080 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:09:58.858947 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:58.861344 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.859244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.18.129.62:45449)
0807 15:09:58.859291 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.859293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.859400 (+ 107us) server_negotiation.cc:407] Connection header received
0807 15:09:58.859460 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.859461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.859489 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.859542 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:58.859620 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.860041 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.860757 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.860925 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.860964 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.861124 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:58.861211 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:58.861263 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.861278 (+ 15us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.861291 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:09:58.861606 10723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:58.863289 10723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:58.865437 10911 tablet_service.cc:1467] Processing CreateTablet for tablet 258b6ca385a54f2990e10f2ccc1bc109 (DEFAULT_TABLE table=test-table [id=1945393d891547f08a587b75b021bc5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:58.865669 10911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258b6ca385a54f2990e10f2ccc1bc109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:58.867018 10849 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Bootstrap starting.
I20260807 15:09:58.867388 10849 tablet_bootstrap.cc:654] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:58.867928 10849 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: No bootstrap required, opened a new log
I20260807 15:09:58.867976 10849 ts_tablet_manager.cc:1397] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:58.868084 10849 raft_consensus.cc:348] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:58.868158 10849 raft_consensus.cc:374] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:58.868216 10849 raft_consensus.cc:729] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8010d4ae6f8495e9e27ece97ed4c7db, State: Initialized, Role: FOLLOWER
I20260807 15:09:58.868263 10849 consensus_queue.cc:260] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:58.868364 10849 ts_tablet_manager.cc:1428] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:09:58.869544 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.865179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43315 (local address 127.0.0.1:48364)
0807 15:09:58.865216 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:58.865227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:58.865260 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:58.865778 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:58.865779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:58.865780 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:58.865822 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.865823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.867130 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.867131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.868247 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.868250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.869316 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.869317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.869380 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.869384 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:58.869431 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:09:58.869478 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 15:09:58.869604 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.865283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.18.129.2:43315)
0807 15:09:58.865547 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.865552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.865564 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:09:58.865604 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.865606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.865622 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.865742 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:58.865888 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.867035 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.868458 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.869191 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.869298 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.869361 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.869469 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.869515 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":133,"threads_started":1}
I20260807 15:09:58.869688 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.865519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.18.129.3:36441)
0807 15:09:58.865710 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.865782 (+ 72us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.865792 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:58.865838 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.865840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.865863 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.865940 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:58.866114 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.867231 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.868670 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.869369 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.869425 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.869478 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.869575 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.869615 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260807 15:09:58.869908 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.865268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:36441 (local address 127.0.0.1:38166)
0807 15:09:58.865362 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:58.865369 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:58.865405 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:58.865937 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:58.865937 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:58.865939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:58.866035 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.866037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.867309 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.867309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.868568 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.868571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.869441 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.869442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.869507 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.869511 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:58.869563 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 15:09:58.869605 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
I20260807 15:09:58.870338 11126 tablet_service.cc:1467] Processing CreateTablet for tablet 258b6ca385a54f2990e10f2ccc1bc109 (DEFAULT_TABLE table=test-table [id=1945393d891547f08a587b75b021bc5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:58.870620 11349 tablet_service.cc:1467] Processing CreateTablet for tablet 258b6ca385a54f2990e10f2ccc1bc109 (DEFAULT_TABLE table=test-table [id=1945393d891547f08a587b75b021bc5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:58.870749 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258b6ca385a54f2990e10f2ccc1bc109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:58.870994 11349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258b6ca385a54f2990e10f2ccc1bc109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:58.874655 11460 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: Bootstrap starting.
I20260807 15:09:58.874732 11459 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Bootstrap starting.
I20260807 15:09:58.875315 11460 tablet_bootstrap.cc:654] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:58.875370 11459 tablet_bootstrap.cc:654] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:58.875816 11460 log.cc:824] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:58.875847 11459 log.cc:824] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:58.876561 11460 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: No bootstrap required, opened a new log
I20260807 15:09:58.876573 11459 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: No bootstrap required, opened a new log
I20260807 15:09:58.876626 11460 ts_tablet_manager.cc:1397] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:58.876631 11459 ts_tablet_manager.cc:1397] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:09:58.878087 11460 raft_consensus.cc:348] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:58.878098 11459 raft_consensus.cc:348] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:58.878232 11460 raft_consensus.cc:374] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:58.878232 11459 raft_consensus.cc:374] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:58.878254 11460 raft_consensus.cc:729] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082e96ba3cf2453590d6a5ae87ac24ff, State: Initialized, Role: FOLLOWER
I20260807 15:09:58.878263 11459 raft_consensus.cc:729] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b80df8a297e46d69cc8c23dcdd1acd6, State: Initialized, Role: FOLLOWER
I20260807 15:09:58.878358 11460 consensus_queue.cc:260] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:58.878398 11459 consensus_queue.cc:260] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:58.878629 11448 heartbeater.cc:503] Master 127.18.129.62:45449 was elected leader, sending a full tablet report...
I20260807 15:09:58.878640 11460 ts_tablet_manager.cc:1428] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:09:58.878679 11459 ts_tablet_manager.cc:1428] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:09:58.878834 11237 heartbeater.cc:503] Master 127.18.129.62:45449 was elected leader, sending a full tablet report...
W20260807 15:09:58.967177 11239 tablet.cc:2369] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:59.029191 11464 raft_consensus.cc:482] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:59.029328 11464 raft_consensus.cc:504] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:59.029726 11464 leader_election.cc:288] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441), c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113)
I20260807 15:09:59.032845 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.029907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34113 (local address 127.18.129.2:55505)
0807 15:09:59.029956 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:59.029969 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:59.030020 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:59.030251 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:59.030252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:59.030255 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:59.030332 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:59.030334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.031089 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.031090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:59.031938 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:59.031940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.032572 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.032573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:59.032675 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.032679 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:59.032741 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 15:09:59.032780 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 15:09:59.032886 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.030054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:55505 (local address 127.18.129.1:34113)
0807 15:09:59.030121 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.030123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.030130 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:09:59.030157 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.030158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.030174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.030230 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:59.030424 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.031020 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.032042 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.032499 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.032560 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.032568 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.032755 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.032794 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:09:59.033030 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.029912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:36441 (local address 127.18.129.2:34313)
0807 15:09:59.030218 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:59.030229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:59.030285 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:59.030511 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:59.030512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:59.030514 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:59.030580 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:59.030581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.031312 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.031313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:59.032172 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:59.032175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.032769 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.032770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:59.032841 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.032845 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:59.032908 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:09:59.032948 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":104,"thread_start_us":128,"threads_started":1}
I20260807 15:09:59.033164 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.030005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:34313 (local address 127.18.129.3:36441)
0807 15:09:59.030054 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.030056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.030294 (+ 238us) server_negotiation.cc:407] Connection header received
0807 15:09:59.030327 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.030328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.030338 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.030400 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:59.030646 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.031205 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.032269 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.032685 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.032750 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.032758 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.033045 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.033100 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:09:59.033164 10948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258b6ca385a54f2990e10f2ccc1bc109" candidate_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" is_pre_election: true
I20260807 15:09:59.033339 10948 raft_consensus.cc:2454] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b80df8a297e46d69cc8c23dcdd1acd6 in term 0.
I20260807 15:09:59.033465 11377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258b6ca385a54f2990e10f2ccc1bc109" candidate_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" is_pre_election: true
I20260807 15:09:59.033578 11060 leader_election.cc:302] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b80df8a297e46d69cc8c23dcdd1acd6, c8010d4ae6f8495e9e27ece97ed4c7db; no voters:
I20260807 15:09:59.033627 11377 raft_consensus.cc:2454] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b80df8a297e46d69cc8c23dcdd1acd6 in term 0.
I20260807 15:09:59.033732 11464 raft_consensus.cc:2781] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:59.033802 11464 raft_consensus.cc:482] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:59.033838 11464 raft_consensus.cc:3037] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:59.034641 11464 raft_consensus.cc:504] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:59.034775 11464 leader_election.cc:288] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [CANDIDATE]: Term 1 election: Requested vote from peers 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441), c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113)
I20260807 15:09:59.034970 11377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258b6ca385a54f2990e10f2ccc1bc109" candidate_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "082e96ba3cf2453590d6a5ae87ac24ff"
I20260807 15:09:59.034976 10948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258b6ca385a54f2990e10f2ccc1bc109" candidate_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db"
I20260807 15:09:59.035046 11377 raft_consensus.cc:3037] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:59.035055 10948 raft_consensus.cc:3037] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:59.035775 11377 raft_consensus.cc:2454] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b80df8a297e46d69cc8c23dcdd1acd6 in term 1.
I20260807 15:09:59.035957 11055 leader_election.cc:302] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 082e96ba3cf2453590d6a5ae87ac24ff, 4b80df8a297e46d69cc8c23dcdd1acd6; no voters:
I20260807 15:09:59.036046 11464 raft_consensus.cc:2781] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:59.036219 11464 raft_consensus.cc:686] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 LEADER]: Becoming Leader. State: Replica: 4b80df8a297e46d69cc8c23dcdd1acd6, State: Running, Role: LEADER
I20260807 15:09:59.036315 10948 raft_consensus.cc:2454] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b80df8a297e46d69cc8c23dcdd1acd6 in term 1.
I20260807 15:09:59.036345 11464 consensus_queue.cc:237] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:09:59.037241 10723 catalog_manager.cc:5495] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2). New cstate: current_term: 1 leader_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } health_report { overall_health: UNKNOWN } } }
I20260807 15:09:59.050299 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.048269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.18.129.2:43315)
0807 15:09:59.048319 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.048320 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.048330 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:09:59.048380 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.048381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.048397 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.048442 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:59.048574 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.049015 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.049711 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.049884 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.049925 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.050041 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:59.050083 (+ 42us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 15:09:59.050220 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48380: token expired
Metrics: {"server-negotiator.queue_time_us":27}
W20260807 15:09:59.050370 11455 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48380: token expired
I20260807 15:09:59.055990 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.050669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.18.129.62:45449)
0807 15:09:59.050713 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.050714 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.050721 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:09:59.050740 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.050741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.050749 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.050787 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.051257 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.051729 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.052337 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.052505 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.052546 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.053612 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:59.053613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:59.053614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:59.053621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:59.054941 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.055242 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.055243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.055243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.055291 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.055522 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.055522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.055522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.055694 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:59.055771 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.055857 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.055923 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:09:59.058679 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.056828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.18.129.2:43315)
0807 15:09:59.056878 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.056881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.056888 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:09:59.056908 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.056909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.056918 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.056960 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:59.057044 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.057472 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.058100 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.058264 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.058303 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.058380 (+ 77us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:59.058484 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:59.058515 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.058604 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.058624 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:09:59.060503 11377 raft_consensus.cc:1261] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 1 FOLLOWER]: Refusing update from remote peer 4b80df8a297e46d69cc8c23dcdd1acd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:59.060518 10948 raft_consensus.cc:1261] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Refusing update from remote peer 4b80df8a297e46d69cc8c23dcdd1acd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:59.060828 11467 consensus_queue.cc:1035] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:59.060951 11464 consensus_queue.cc:1035] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:59.063267 11469 mvcc.cc:205] Tried to move back new op lower bound from 7315928674549473280 to 7315928674453639168. Current Snapshot: MvccSnapshot[applied={T|T < 7315928674549473280}]
I20260807 15:09:59.065503 10720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:59.065593 10720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:59.066319 10720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Sending DeleteTablet for 3 replicas of tablet 258b6ca385a54f2990e10f2ccc1bc109
I20260807 15:09:59.066711 10911 tablet_service.cc:1514] Processing DeleteTablet for tablet 258b6ca385a54f2990e10f2ccc1bc109 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:59 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:41444
I20260807 15:09:59.066733 11126 tablet_service.cc:1514] Processing DeleteTablet for tablet 258b6ca385a54f2990e10f2ccc1bc109 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:59 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:48364
I20260807 15:09:59.066807 11032 tablet_replica.cc:330] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: stopping tablet replica
I20260807 15:09:59.066789 11349 tablet_service.cc:1514] Processing DeleteTablet for tablet 258b6ca385a54f2990e10f2ccc1bc109 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:59 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:38166
I20260807 15:09:59.066891 11032 raft_consensus.cc:2229] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:59.066936 11032 raft_consensus.cc:2258] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:59.067065 11489 tablet_replica.cc:330] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: stopping tablet replica
I20260807 15:09:59.067174 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 10787
I20260807 15:09:59.067193 11489 raft_consensus.cc:2229] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:59.067291 11489 raft_consensus.cc:2258] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:59.071420 11487 tablet_replica.cc:330] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: stopping tablet replica
I20260807 15:09:59.071535 11487 raft_consensus.cc:2229] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:59.071673 11487 raft_consensus.cc:2258] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:59.073828 11487 ts_tablet_manager.cc:1905] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:59.075443 11487 ts_tablet_manager.cc:1918] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:59.075518 11487 log.cc:1196] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/258b6ca385a54f2990e10f2ccc1bc109
I20260807 15:09:59.075807 11487 ts_tablet_manager.cc:1939] T 258b6ca385a54f2990e10f2ccc1bc109 P 4b80df8a297e46d69cc8c23dcdd1acd6: Deleting consensus metadata
I20260807 15:09:59.076248 10691 catalog_manager.cc:4841] TS 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315): tablet 258b6ca385a54f2990e10f2ccc1bc109 (table test-table [id=1945393d891547f08a587b75b021bc5d]) successfully deleted
I20260807 15:09:59.079545 11489 ts_tablet_manager.cc:1905] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:59.081365 11489 ts_tablet_manager.cc:1918] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:59.081507 11489 log.cc:1196] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/258b6ca385a54f2990e10f2ccc1bc109
I20260807 15:09:59.081827 11489 ts_tablet_manager.cc:1939] T 258b6ca385a54f2990e10f2ccc1bc109 P 082e96ba3cf2453590d6a5ae87ac24ff: Deleting consensus metadata
I20260807 15:09:59.082331 10691 catalog_manager.cc:4841] TS 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441): tablet 258b6ca385a54f2990e10f2ccc1bc109 (table test-table [id=1945393d891547f08a587b75b021bc5d]) successfully deleted
W20260807 15:09:59.085659 10695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:34113 (error 108)
I20260807 15:09:59.086014 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 11022
I20260807 15:09:59.086218 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.086075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34113 (local address 127.0.0.1:41452)
0807 15:09:59.086140 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:59.086179 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 15:09:59.091010 10695 catalog_manager.cc:4584] TS c8010d4ae6f8495e9e27ece97ed4c7db (127.18.129.1:34113): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 258b6ca385a54f2990e10f2ccc1bc109: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34113: connect: Connection refused (error 111)
I20260807 15:09:59.095121 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 11246
I20260807 15:09:59.104439 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 10668
I20260807 15:10:19.114686 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45449
--webserver_interface=127.18.129.62
--webserver_port=35349
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:45449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:19.199136 11491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:19.199344 11491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:19.199373 11491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:19.200476 11491 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:10:19.200510 11491 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:10:19.200521 11491 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:10:19.200532 11491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:19.200543 11491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:19.200562 11491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:19.200572 11491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:19.200582 11491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:19.202006 11491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:45449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:45449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:19.202203 11491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:19.202397 11491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:19.207573 11505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:19.207580 11500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:19.207612 11491 server_base.cc:1034] running on GCE node
W20260807 15:10:19.207651 11502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:19.208099 11491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:19.208385 11491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:19.209539 11491 hybrid_clock.cc:648] HybridClock initialized: now 1786115419209521 us; error 35 us; skew 500 ppm
Aug 07 15:10:19 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:19.214937 11491 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:10:19.215415 11491 webserver.cc:460] Webserver started at http://127.18.129.62:35349/ using document root <none> and password file <none>
I20260807 15:10:19.215642 11491 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:19.215694 11491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:19.217002 11491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:19.217653 11513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:19.217809 11491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:19.217900 11491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "7b6eaf69913b4299bbf7968911821165"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "7229eae38041decccf8539b43b97726e"
server_key_iv: "cb4f639e02ffde552f89e7a8fcdd8a9d"
server_key_version: "encryptionkey@0"
I20260807 15:10:19.218263 11491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:19.249619 11491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:19.251730 11491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:19.251883 11491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:19.255966 11491 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:45449
I20260807 15:10:19.256031 11595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:45449 every 8 connection(s)
I20260807 15:10:19.256402 11491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:19.257979 11597 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:10:19.258648 11597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Bootstrap starting.
I20260807 15:10:19.260818 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 11491
I20260807 15:10:19.261087 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:34113
--local_ip_for_outbound_sockets=127.18.129.1
--tserver_master_addrs=127.18.129.62:45449
--webserver_port=44275
--webserver_interface=127.18.129.1
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:10:19.268215 11597 log.cc:824] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:19.270466 11597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:10:19.270768 11597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Bootstrap complete.
I20260807 15:10:19.272982 11597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:10:19.273218 11597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6eaf69913b4299bbf7968911821165, State: Initialized, Role: FOLLOWER
I20260807 15:10:19.273377 11597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:10:19.273445 11597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:19.273487 11597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:19.273532 11597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:10:19.274526 11597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:10:19.274622 11597 leader_election.cc:302] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b6eaf69913b4299bbf7968911821165; no voters:
I20260807 15:10:19.274799 11597 leader_election.cc:288] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:10:19.274873 11601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:10:19.275036 11601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [term 3 LEADER]: Becoming Leader. State: Replica: 7b6eaf69913b4299bbf7968911821165, State: Running, Role: LEADER
I20260807 15:10:19.275125 11597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:19.275143 11601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } }
I20260807 15:10:19.275406 11601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7b6eaf69913b4299bbf7968911821165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } } }
I20260807 15:10:19.275454 11605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b6eaf69913b4299bbf7968911821165. Latest consensus state: current_term: 3 leader_uuid: "7b6eaf69913b4299bbf7968911821165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6eaf69913b4299bbf7968911821165" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 45449 } } }
I20260807 15:10:19.275576 11601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:19.275818 11605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:19.276253 11616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:19.277038 11616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:19.277136 11616 catalog_manager.cc:1259] Loaded cluster ID: 0ed1c2b448e146e299b71f149fe02d63
I20260807 15:10:19.277170 11616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:19.277937 11616 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:19.278247 11616 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Loaded TSK: 0
I20260807 15:10:19.278316 11616 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b6eaf69913b4299bbf7968911821165: Loaded TSK: 1
F20260807 15:10:19.278563 11616 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6cdf9969ad google::LogMessage::Fail() at ??:0
@ 0x7f6cdf99988a google::LogMessage::SendToLog() at ??:0
@ 0x7f6cdf9964de google::LogMessage::Flush() at ??:0
@ 0x7f6cdf997a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6ce2103104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6ce2100d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6ce78f01cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6ce78f048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6ce78f0594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6ce78b6c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6ce78c0ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6ce18640d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6ce1859954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6ce26bd6db start_thread at ??:0
@ 0x7f6cde49461f clone at ??:0
W20260807 15:10:19.358563 11599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:19.358774 11599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:19.358805 11599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:19.360000 11599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:19.360045 11599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:19.360080 11599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:10:19.361549 11599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:34113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=44275
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:19.361783 11599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:19.362013 11599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:19.367265 11667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:19.367255 11665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:19.367266 11663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:19.367585 11599 server_base.cc:1034] running on GCE node
I20260807 15:10:19.367851 11599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:19.368110 11599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:19.369261 11599 hybrid_clock.cc:648] HybridClock initialized: now 1786115419369244 us; error 34 us; skew 500 ppm
Aug 07 15:10:19 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:19.372107 11599 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:10:19.372589 11599 webserver.cc:460] Webserver started at http://127.18.129.1:44275/ using document root <none> and password file <none>
I20260807 15:10:19.372797 11599 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:19.372848 11599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:19.374326 11599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:19.375139 11689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:19.375324 11599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:19.375411 11599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "c8010d4ae6f8495e9e27ece97ed4c7db"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "ee16917db464177d3ee80274f3ab8a52"
server_key_iv: "2d3dfe19b8182e09b50091520cf8a0c6"
server_key_version: "encryptionkey@0"
I20260807 15:10:19.375793 11599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:19.393368 11599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:19.395704 11599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:19.395844 11599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:19.396065 11599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:19.396514 11698 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:10:19.398615 11599 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:10:19.398664 11599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:10:19.398698 11599 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:10:19.399369 11599 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:10:19.399402 11599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:10:19.399506 11698 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Bootstrap starting.
I20260807 15:10:19.406073 11599 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:34113
I20260807 15:10:19.406152 11860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:34113 every 8 connection(s)
I20260807 15:10:19.406502 11599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:19.407291 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 11599
I20260807 15:10:19.407636 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:43315
--local_ip_for_outbound_sockets=127.18.129.2
--tserver_master_addrs=127.18.129.62:45449
--webserver_port=46265
--webserver_interface=127.18.129.2
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:10:19.411317 11698 log.cc:824] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:19.414317 11698 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:10:19.414695 11698 tablet_bootstrap.cc:492] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Bootstrap complete.
I20260807 15:10:19.415995 11698 ts_tablet_manager.cc:1397] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.012s
I20260807 15:10:19.417569 11698 raft_consensus.cc:348] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:10:19.417933 11698 raft_consensus.cc:729] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8010d4ae6f8495e9e27ece97ed4c7db, State: Initialized, Role: FOLLOWER
I20260807 15:10:19.418135 11698 consensus_queue.cc:260] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:10:19.418465 11698 ts_tablet_manager.cc:1428] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260807 15:10:19.493577 11867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:19.493799 11867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:19.493831 11867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:19.494628 11866 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:19.407267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:58175)
0807 15:10:19.407486 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.407539 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:19.408142 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:19.494421 (+ 86279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":108,"threads_started":1}
W20260807 15:10:19.494982 11864 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:45449 (0 consecutive failures): Network error: Failed to ping master at 127.18.129.62:45449: Client connection negotiation failed: client connection to 127.18.129.62:45449: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 15:10:19.495249 11867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:19.495307 11867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:19.495347 11867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:10:19.495344 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.495207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:48103)
0807 15:10:19.495259 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.495315 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 15:10:19.495671 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.495575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:53871)
0807 15:10:19.495616 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.495646 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 15:10:19.495792 11864 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:10:19.497041 11867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:43315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=46265
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:19.497246 11867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:19.497434 11867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:19.502377 11879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:19.502370 11876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:19.502453 11867 server_base.cc:1034] running on GCE node
W20260807 15:10:19.502370 11877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:19.502893 11867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:19.503144 11867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:19.504269 11867 hybrid_clock.cc:648] HybridClock initialized: now 1786115419504261 us; error 27 us; skew 500 ppm
Aug 07 15:10:19 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:19.506721 11867 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:10:19.507169 11867 webserver.cc:460] Webserver started at http://127.18.129.2:46265/ using document root <none> and password file <none>
I20260807 15:10:19.507357 11867 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:19.507397 11867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:19.508651 11867 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:19.509214 11893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:19.509359 11867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:10:19.509446 11867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "4b80df8a297e46d69cc8c23dcdd1acd6"
format_stamp: "Formatted at 2026-08-07 15:09:37 on dist-test-slave-5bn2"
server_key: "27897afc009ef5444f3b390ac0719dea"
server_key_iv: "0299f4ec68a3c4ba445889787359cd46"
server_key_version: "encryptionkey@0"
I20260807 15:10:19.509799 11867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:19.522099 11867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:19.525816 11867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:19.525964 11867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:19.526176 11867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:19.526571 11867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:19.526620 11867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:19.526651 11867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:19.526679 11867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:19.534225 11867 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:43315
I20260807 15:10:19.534284 12081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:43315 every 8 connection(s)
I20260807 15:10:19.534715 11867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:19.535028 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 11867
I20260807 15:10:19.535320 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:36441
--local_ip_for_outbound_sockets=127.18.129.3
--tserver_master_addrs=127.18.129.62:45449
--webserver_port=43055
--webserver_interface=127.18.129.3
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:10:19.538051 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.537508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:45131)
0807 15:10:19.537708 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.537837 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":103,"threads_started":1}
W20260807 15:10:19.538466 12082 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:45449 (0 consecutive failures): Network error: Failed to ping master at 127.18.129.62:45449: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
I20260807 15:10:19.538748 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.538664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:35893)
0807 15:10:19.538696 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.538723 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260807 15:10:19.539096 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.539007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:47899)
0807 15:10:19.539033 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.539072 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260807 15:10:19.539196 12082 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:10:19.624604 12084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:19.624818 12084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:19.624842 12084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:19.626093 12084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:19.626140 12084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:19.626168 12084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:10:19.627692 12084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:36441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=43055
--tserver_master_addrs=127.18.129.62:45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:19.627909 12084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:19.628140 12084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:19.633287 12095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:19.633332 12092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:19.633312 12098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:19.633400 12084 server_base.cc:1034] running on GCE node
I20260807 15:10:19.633790 12084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:19.634097 12084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:19.635260 12084 hybrid_clock.cc:648] HybridClock initialized: now 1786115419635238 us; error 34 us; skew 500 ppm
Aug 07 15:10:19 dist-test-slave-5bn2 krb5kdc[9830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:19.637917 12084 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:10:19.638334 12084 webserver.cc:460] Webserver started at http://127.18.129.3:43055/ using document root <none> and password file <none>
I20260807 15:10:19.638588 12084 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:19.638644 12084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:19.639986 12084 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:19.640647 12105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:19.640810 12084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:19.640897 12084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "082e96ba3cf2453590d6a5ae87ac24ff"
format_stamp: "Formatted at 2026-08-07 15:09:38 on dist-test-slave-5bn2"
server_key: "d3037b2e9716e1226ede6bff386509d5"
server_key_iv: "9c4f6d8726498949fe75fa4795f6170e"
server_key_version: "encryptionkey@0"
I20260807 15:10:19.641235 12084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:19.653599 12084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:19.655879 12084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:19.655997 12084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:19.656283 12084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:19.656584 12084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:19.656618 12084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:19.656647 12084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:19.656672 12084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:19.662395 12084 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:36441
I20260807 15:10:19.662530 12277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:36441 every 8 connection(s)
I20260807 15:10:19.662808 12084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:19.663867 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.663488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:41393)
0807 15:10:19.663655 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.663727 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
W20260807 15:10:19.664255 12281 heartbeater.cc:644] Failed to heartbeat to 127.18.129.62:45449 (0 consecutive failures): Network error: Failed to ping master at 127.18.129.62:45449: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
I20260807 15:10:19.664499 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.664413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:34339)
0807 15:10:19.664456 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.664483 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 15:10:19.664713 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.664655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:41949)
0807 15:10:19.664683 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.664702 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 15:10:19.664808 12281 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:10:19.671969 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 12084
I20260807 15:10:20.496417 12285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:20.496169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:48807)
0807 15:10:20.496301 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:20.496345 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260807 15:10:20.539757 12286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:20.539502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:45465)
0807 15:10:20.539653 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:20.539696 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":61,"threads_started":1}
I20260807 15:10:20.665366 12288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:20.665132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:55589)
0807 15:10:20.665271 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:20.665312 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":57,"threads_started":1}
I20260807 15:10:21.053153 12289 raft_consensus.cc:482] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:21.053316 12289 raft_consensus.cc:504] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:10:21.053642 12289 leader_election.cc:288] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315), 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441)
Aug 07 15:10:21 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.3@KRBTEST.COM
Aug 07 15:10:21 dist-test-slave-5bn2 krb5kdc[9830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.2@KRBTEST.COM
I20260807 15:10:21.063413 12292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.053949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:36441 (local address 127.18.129.1:59837)
0807 15:10:21.054056 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:21.054065 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:21.054115 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:21.054919 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:21.054920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:21.055312 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:21.055511 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.055514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.056638 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.056639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.057459 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.057462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.058019 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.058020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.058128 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.058667 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:21.058674 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:21.060139 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:21.061952 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:21.061959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:21.061961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:21.062186 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:21.062413 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:21.062414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:21.062414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:21.062523 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:21.062886 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:21.062893 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:21.063028 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 15:10:21.063114 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:10:21.063285 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":5,"thread_start_us":57,"threads_started":1}
I20260807 15:10:21.063517 12291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.053913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:59619 (local address 127.18.129.2:43315)
0807 15:10:21.054148 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:21.054161 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:21.054187 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:10:21.054657 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:21.054659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:21.054701 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:21.054812 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:21.055621 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.056551 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.057541 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.057932 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.058033 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.060610 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:21.060616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:21.060616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:21.060629 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:21.062193 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:21.062509 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:21.062512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:21.062512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:21.062599 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:21.062775 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:21.062776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:21.062776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:21.063043 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:21.063128 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:21.063288 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 15:10:21.063414 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260807 15:10:21.063706 12293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.054074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:59837 (local address 127.18.129.3:36441)
0807 15:10:21.054228 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:21.054236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:21.054264 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:10:21.054698 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:21.054700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:21.054736 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:21.054891 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:21.055624 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.056544 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.057541 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.057934 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.058033 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.060242 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:21.060246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:21.060246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:21.060257 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:21.061876 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:21.062261 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:21.062264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:21.062265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:21.062354 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:21.062580 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:21.062580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:21.062581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:21.062807 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:21.062908 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:21.063175 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 15:10:21.063552 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260807 15:10:21.063630 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.053791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:43315 (local address 127.18.129.1:59619)
0807 15:10:21.053896 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:21.053908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:21.053979 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:21.054912 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:21.054915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:21.055310 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:21.055507 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.055513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.056662 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.056663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.057461 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.057464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.058015 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.058016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.058129 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.058719 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:21.058724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:21.060494 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:21.062276 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:21.062277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:21.062277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:21.062438 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:21.062659 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:21.062660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:21.062660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:21.062720 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:21.063110 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:21.063111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:21.063155 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 15:10:21.063218 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:10:21.063353 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":90,"thread_start_us":53,"threads_started":1}
W20260807 15:10:21.064740 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
W20260807 15:10:21.064815 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
I20260807 15:10:21.064852 11691 leader_election.cc:302] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8010d4ae6f8495e9e27ece97ed4c7db; no voters: 082e96ba3cf2453590d6a5ae87ac24ff, 4b80df8a297e46d69cc8c23dcdd1acd6
I20260807 15:10:21.064949 12289 raft_consensus.cc:2726] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:10:21.496995 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.496860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:38037)
0807 15:10:21.496910 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:21.496951 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 15:10:21.540481 12294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.540205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:57963)
0807 15:10:21.540361 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:21.540426 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":63,"threads_started":1}
I20260807 15:10:21.666023 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.665803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:33287)
0807 15:10:21.665935 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:21.665979 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260807 15:10:22.497735 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:22.497477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:38685)
0807 15:10:22.497614 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:22.497661 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260807 15:10:22.541093 12298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:22.540878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:59081)
0807 15:10:22.540986 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:22.541045 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260807 15:10:22.666723 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:22.666473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:51701)
0807 15:10:22.666639 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:22.666675 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260807 15:10:23.053793 12300 raft_consensus.cc:482] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:23.053910 12300 raft_consensus.cc:504] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:10:23.054075 12300 leader_election.cc:288] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315), 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441)
W20260807 15:10:23.054514 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
W20260807 15:10:23.054584 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
I20260807 15:10:23.054613 11691 leader_election.cc:302] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8010d4ae6f8495e9e27ece97ed4c7db; no voters: 082e96ba3cf2453590d6a5ae87ac24ff, 4b80df8a297e46d69cc8c23dcdd1acd6
I20260807 15:10:23.054697 12300 raft_consensus.cc:2726] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:10:23.498428 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:23.498194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:47791)
0807 15:10:23.498332 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:23.498374 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260807 15:10:23.541834 12302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:23.541575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:41555)
0807 15:10:23.541717 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:23.541760 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260807 15:10:23.667433 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:23.667213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:56869)
0807 15:10:23.667334 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:23.667376 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260807 15:10:24.499238 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:24.498955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:41955)
0807 15:10:24.499139 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:24.499184 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260807 15:10:24.542505 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:24.542270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:51629)
0807 15:10:24.542410 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:24.542452 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260807 15:10:24.668095 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:24.667862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:43737)
0807 15:10:24.668003 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:24.668047 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260807 15:10:25.499965 12310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:25.499726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:52609)
0807 15:10:25.499868 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:25.499911 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260807 15:10:25.543149 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:25.542947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:39107)
0807 15:10:25.543061 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:25.543100 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260807 15:10:25.668766 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:25.668543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:56043)
0807 15:10:25.668681 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:25.668718 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
I20260807 15:10:26.035549 12313 raft_consensus.cc:482] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:26.035697 12313 raft_consensus.cc:504] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:10:26.035856 12313 leader_election.cc:288] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315), 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441)
W20260807 15:10:26.036262 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
W20260807 15:10:26.036331 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
I20260807 15:10:26.036361 11691 leader_election.cc:302] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8010d4ae6f8495e9e27ece97ed4c7db; no voters: 082e96ba3cf2453590d6a5ae87ac24ff, 4b80df8a297e46d69cc8c23dcdd1acd6
I20260807 15:10:26.036448 12313 raft_consensus.cc:2726] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:10:26.500646 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:26.500418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:36817)
0807 15:10:26.500544 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:26.500594 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260807 15:10:26.543774 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:26.543535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:49575)
0807 15:10:26.543685 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:26.543726 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260807 15:10:26.669454 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:26.669218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:52137)
0807 15:10:26.669366 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:26.669407 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260807 15:10:27.501363 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:27.501121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:48033)
0807 15:10:27.501264 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:27.501310 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260807 15:10:27.544488 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:27.544253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:38663)
0807 15:10:27.544395 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:27.544437 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260807 15:10:27.670188 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:27.669939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:52563)
0807 15:10:27.670086 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:27.670141 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260807 15:10:28.502044 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:28.501814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:60067)
0807 15:10:28.501949 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:28.501991 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260807 15:10:28.545130 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:28.544906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:41747)
0807 15:10:28.545029 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:28.545071 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260807 15:10:28.670904 12328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:28.670640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:48191)
0807 15:10:28.670777 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:28.670820 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260807 15:10:29.502753 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:29.502500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:55611)
0807 15:10:29.502665 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:29.502701 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260807 15:10:29.545806 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:29.545553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:48127)
0807 15:10:29.545719 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:29.545756 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260807 15:10:29.606415 12333 raft_consensus.cc:482] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:29.606524 12333 raft_consensus.cc:504] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:10:29.606695 12333 leader_election.cc:288] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315), 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441)
W20260807 15:10:29.607082 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
W20260807 15:10:29.607167 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
I20260807 15:10:29.607200 11691 leader_election.cc:302] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8010d4ae6f8495e9e27ece97ed4c7db; no voters: 082e96ba3cf2453590d6a5ae87ac24ff, 4b80df8a297e46d69cc8c23dcdd1acd6
I20260807 15:10:29.607280 12333 raft_consensus.cc:2726] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:10:29.671733 12335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:29.671464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:35121)
0807 15:10:29.671641 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:29.671682 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260807 15:10:30.503477 12336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:30.503235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:40755)
0807 15:10:30.503381 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:30.503425 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260807 15:10:30.546470 12337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:30.546253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:49877)
0807 15:10:30.546382 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:30.546420 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260807 15:10:30.672487 12338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:30.672244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:35783)
0807 15:10:30.672392 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:30.672435 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
I20260807 15:10:31.504113 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:31.503898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:57159)
0807 15:10:31.504030 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:31.504064 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260807 15:10:31.547147 12342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:31.546925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:50527)
0807 15:10:31.547063 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:31.547099 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260807 15:10:31.673173 12343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:31.672929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:53159)
0807 15:10:31.673086 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:31.673124 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260807 15:10:32.504865 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.504589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:53245)
0807 15:10:32.504763 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:32.504810 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":70,"threads_started":1}
I20260807 15:10:32.547797 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.547578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:53961)
0807 15:10:32.547712 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:32.547746 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
I20260807 15:10:32.673826 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.673595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:45905)
0807 15:10:32.673736 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:32.673777 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260807 15:10:32.674527 12348 raft_consensus.cc:482] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:32.674631 12348 raft_consensus.cc:504] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b80df8a297e46d69cc8c23dcdd1acd6" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 43315 } } peers { permanent_uuid: "082e96ba3cf2453590d6a5ae87ac24ff" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 36441 } } peers { permanent_uuid: "c8010d4ae6f8495e9e27ece97ed4c7db" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34113 } }
I20260807 15:10:32.674786 12348 leader_election.cc:288] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315), 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441)
W20260807 15:10:32.675194 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 082e96ba3cf2453590d6a5ae87ac24ff (127.18.129.3:36441): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
W20260807 15:10:32.675258 11691 leader_election.cc:341] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4b80df8a297e46d69cc8c23dcdd1acd6 (127.18.129.2:43315): Not found: Tablet not found: 258b6ca385a54f2990e10f2ccc1bc109
I20260807 15:10:32.675273 11691 leader_election.cc:302] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8010d4ae6f8495e9e27ece97ed4c7db; no voters: 082e96ba3cf2453590d6a5ae87ac24ff, 4b80df8a297e46d69cc8c23dcdd1acd6
I20260807 15:10:32.675362 12348 raft_consensus.cc:2726] T 258b6ca385a54f2990e10f2ccc1bc109 P c8010d4ae6f8495e9e27ece97ed4c7db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:10:33.505609 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:33.505357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:41151)
0807 15:10:33.505511 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:33.505555 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260807 15:10:33.548590 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:33.548302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:45339)
0807 15:10:33.548499 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:33.548540 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
I20260807 15:10:33.674607 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:33.674289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:51287)
0807 15:10:33.674493 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:33.674537 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":142,"threads_started":1}
I20260807 15:10:34.506474 12358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:34.506192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.1:36253)
0807 15:10:34.506318 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:34.506357 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260807 15:10:34.549362 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:34.549124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.2:34981)
0807 15:10:34.549274 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:34.549312 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260807 15:10:34.675364 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:34.675115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:45449 (local address 127.18.129.3:33875)
0807 15:10:34.675276 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:34.675316 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.62:45449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260807 15:10:34.680001 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 11599
I20260807 15:10:34.689167 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 11867
I20260807 15:10:34.700567 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 12084
2026-08-07T15:10:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58489 ms)
[----------] 30 tests from SecurityITest (237360 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:34 dist-test-slave-5bn2 krb5kdc[12367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:34 dist-test-slave-5bn2 krb5kdc[12367](info): set up 2 sockets
Aug 07 15:10:34 dist-test-slave-5bn2 krb5kdc[12367](info): commencing operation
krb5kdc: starting...
W20260807 15:10:35.841442 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:35 dist-test-slave-5bn2 krb5kdc[12367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:10:35.969281 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34337
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:34337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:36.049727 12388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:36.049938 12388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:36.049968 12388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:36.051142 12388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:36.051182 12388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:36.051200 12388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:36.051218 12388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:36.051232 12388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:36.052685 12388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:34337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:36.052932 12388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:36.053160 12388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:36.057920 12394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:36.058113 12388 server_base.cc:1034] running on GCE node
W20260807 15:10:36.057996 12395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:36.057936 12398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:36.058470 12388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:36.058759 12388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:36.059919 12388 hybrid_clock.cc:648] HybridClock initialized: now 1786115436059895 us; error 33 us; skew 500 ppm
Aug 07 15:10:36 dist-test-slave-5bn2 krb5kdc[12367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:36.062544 12388 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:10:36.063060 12388 webserver.cc:460] Webserver started at http://127.18.129.62:38455/ using document root <none> and password file <none>
I20260807 15:10:36.063280 12388 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:36.063333 12388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:36.063479 12388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:36.064592 12388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "0bd9e07ab8904fb0bc14d2716d421100"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "8e5141e129b0a45a8c7fba978abf5943"
server_key_iv: "c71f7872df91d31576c437664fa9a2cb"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.064970 12388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "0bd9e07ab8904fb0bc14d2716d421100"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "8e5141e129b0a45a8c7fba978abf5943"
server_key_iv: "c71f7872df91d31576c437664fa9a2cb"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.066341 12388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:36.067107 12409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.067270 12388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:36.067366 12388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "0bd9e07ab8904fb0bc14d2716d421100"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "8e5141e129b0a45a8c7fba978abf5943"
server_key_iv: "c71f7872df91d31576c437664fa9a2cb"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.067437 12388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:36.093412 12388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:36.095674 12388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:36.095788 12388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:36.099829 12388 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:34337
I20260807 15:10:36.099869 12493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:34337 every 8 connection(s)
I20260807 15:10:36.100224 12388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:36.101110 12494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:36.103488 12494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100: Bootstrap starting.
I20260807 15:10:36.104074 12494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:36.104394 12494 log.cc:824] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:36.104444 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 12388
I20260807 15:10:36.104527 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:36.104724 18948 external_mini_cluster.cc:1428] Setting key a47b6bcb039a8e70a65590bda0957369
I20260807 15:10:36.105237 12494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100: No bootstrap required, opened a new log
I20260807 15:10:36.107401 12494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd9e07ab8904fb0bc14d2716d421100" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34337 } }
I20260807 15:10:36.107582 12494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:36.107620 12494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd9e07ab8904fb0bc14d2716d421100, State: Initialized, Role: FOLLOWER
I20260807 15:10:36.107761 12494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd9e07ab8904fb0bc14d2716d421100" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34337 } }
I20260807 15:10:36.107837 12494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:36.107874 12494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:36.107916 12494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:36.108595 12494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd9e07ab8904fb0bc14d2716d421100" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34337 } }
I20260807 15:10:36.108704 12494 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd9e07ab8904fb0bc14d2716d421100; no voters:
I20260807 15:10:36.108855 12494 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:36.108896 12499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:36.109087 12499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [term 1 LEADER]: Becoming Leader. State: Replica: 0bd9e07ab8904fb0bc14d2716d421100, State: Running, Role: LEADER
I20260807 15:10:36.109097 12494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:36.109218 12499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd9e07ab8904fb0bc14d2716d421100" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34337 } }
I20260807 15:10:36.109704 12500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bd9e07ab8904fb0bc14d2716d421100" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd9e07ab8904fb0bc14d2716d421100" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34337 } } }
I20260807 15:10:36.109777 12500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:36.109982 12501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd9e07ab8904fb0bc14d2716d421100. Latest consensus state: current_term: 1 leader_uuid: "0bd9e07ab8904fb0bc14d2716d421100" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd9e07ab8904fb0bc14d2716d421100" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34337 } } }
I20260807 15:10:36.110364 12509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:36.110384 12501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:36.110913 12509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:36.112569 12509 catalog_manager.cc:1347] Generated new cluster ID: 6ed41e7c558d43c19f5f6b32a27e02eb
I20260807 15:10:36.112622 12509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:10:36 dist-test-slave-5bn2 krb5kdc[12367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:36.118535 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.105292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.18.129.62:34337)
0807 15:10:36.105491 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.105501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.105535 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:10:36.106565 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.106575 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.106649 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.106795 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:36.107408 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.109032 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.110184 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.112082 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.112183 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.114819 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:36.114824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:36.114824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:36.114836 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:36.116707 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.117083 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.117086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.117086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.117179 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.117338 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.117339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.117339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.117608 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:36.117736 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.118198 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.118325 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
I20260807 15:10:36.122188 12509 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:36.122689 12509 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:36.128085 12509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100: Generated new TSK 0
I20260807 15:10:36.128240 12509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:10:36.159796 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34337
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:36.242089 12530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:36.242300 12530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:36.242321 12530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:36.243506 12530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:36.243551 12530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:36.243579 12530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:10:36.245087 12530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:36.245312 12530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:36.245534 12530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:36.250833 12539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:36.250816 12536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:36.250803 12537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:36.250870 12530 server_base.cc:1034] running on GCE node
I20260807 15:10:36.251268 12530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:36.251541 12530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:36.252699 12530 hybrid_clock.cc:648] HybridClock initialized: now 1786115436252682 us; error 31 us; skew 500 ppm
Aug 07 15:10:36 dist-test-slave-5bn2 krb5kdc[12367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:36.255330 12530 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:10:36.255753 12530 webserver.cc:460] Webserver started at http://127.18.129.1:43071/ using document root <none> and password file <none>
I20260807 15:10:36.255962 12530 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:36.256013 12530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:36.256143 12530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:36.257190 12530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "070a905fe8fe458b8793076182fac692"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "aae9bb5a85eb970fa05e96292e3dd014"
server_key_iv: "4a318a83c5b6a918d42e27c4f14a062e"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.257525 12530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "070a905fe8fe458b8793076182fac692"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "aae9bb5a85eb970fa05e96292e3dd014"
server_key_iv: "4a318a83c5b6a918d42e27c4f14a062e"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.258791 12530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:10:36.259513 12547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.259689 12530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:36.259780 12530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "070a905fe8fe458b8793076182fac692"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "aae9bb5a85eb970fa05e96292e3dd014"
server_key_iv: "4a318a83c5b6a918d42e27c4f14a062e"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.259838 12530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:36.282384 12530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:36.284732 12530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:36.284855 12530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:36.285053 12530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:36.285429 12530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:36.285471 12530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.285495 12530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:36.285519 12530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.291138 12530 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:35539
I20260807 15:10:36.291214 12719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:35539 every 8 connection(s)
I20260807 15:10:36.291515 12530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:36.295362 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 12530
I20260807 15:10:36.295456 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:36.295651 18948 external_mini_cluster.cc:1428] Setting key 80c39170afc1bd258a74bc030417fa3e
Aug 07 15:10:36 dist-test-slave-5bn2 krb5kdc[12367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:36.300884 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.292212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:41799 (local address 127.18.129.62:34337)
0807 15:10:36.292277 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.292279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.292479 (+ 200us) server_negotiation.cc:407] Connection header received
0807 15:10:36.292981 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.292982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.292992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.293037 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:36.294017 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.294532 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.295584 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.295860 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.295926 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.298288 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:36.298289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:36.298290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:36.298297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:36.299595 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.299975 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.299976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.299976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.300038 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.300265 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.300266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.300266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.300376 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:36.300432 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.300707 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.300793 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:10:36.301110 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.292185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34337 (local address 127.18.129.1:41799)
0807 15:10:36.292360 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.292397 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.292887 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.293111 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.293115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.293735 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:36.293941 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.293948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.294618 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.294619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.295502 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.295505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.295949 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.295950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.296082 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.296581 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:36.296609 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:36.298195 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:36.299676 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:36.299683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:36.299686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:36.299905 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:36.300096 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:36.300097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:36.300098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:36.300206 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:36.300433 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:36.300441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:36.300587 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.300674 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.300835 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260807 15:10:36.301560 12722 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34337
I20260807 15:10:36.301649 12722 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:36.301820 12722 heartbeater.cc:511] Master 127.18.129.62:34337 requested a full tablet report, sending...
I20260807 15:10:36.302488 12437 ts_manager.cc:194] Registered new tserver with Master: 070a905fe8fe458b8793076182fac692 (127.18.129.1:35539)
I20260807 15:10:36.303215 12437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:41799
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:10:36.322074 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34337
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:36.404350 12732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:36.404559 12732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:36.404580 12732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:36.405661 12732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:36.405692 12732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:36.405716 12732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:10:36.407153 12732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:36.407352 12732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:36.407527 12732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:36.412643 12742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:36.412757 12732 server_base.cc:1034] running on GCE node
W20260807 15:10:36.412709 12745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:36.412832 12743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:36.413081 12732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:36.413304 12732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:36.414440 12732 hybrid_clock.cc:648] HybridClock initialized: now 1786115436414428 us; error 28 us; skew 500 ppm
Aug 07 15:10:36 dist-test-slave-5bn2 krb5kdc[12367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:36.416916 12732 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:10:36.417316 12732 webserver.cc:460] Webserver started at http://127.18.129.2:39905/ using document root <none> and password file <none>
I20260807 15:10:36.417488 12732 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:36.417529 12732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:36.417620 12732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:36.418599 12732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "71cb979e6fa0407c9bfd549bdad549af"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "62279fe90a3554723f8d346ed5b553a2"
server_key_iv: "a7b3b0bb131e4b9877291c6472e7de2b"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.419025 12732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "71cb979e6fa0407c9bfd549bdad549af"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "62279fe90a3554723f8d346ed5b553a2"
server_key_iv: "a7b3b0bb131e4b9877291c6472e7de2b"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.420317 12732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:36.421005 12756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.421162 12732 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:10:36.421247 12732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "71cb979e6fa0407c9bfd549bdad549af"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "62279fe90a3554723f8d346ed5b553a2"
server_key_iv: "a7b3b0bb131e4b9877291c6472e7de2b"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.421306 12732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:36.436267 12732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:36.438369 12732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:36.438508 12732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:36.438735 12732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:36.439097 12732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:36.439136 12732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.439163 12732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:36.439180 12732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.444733 12732 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:37295
I20260807 15:10:36.444802 12919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:37295 every 8 connection(s)
I20260807 15:10:36.445096 12732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:36.447685 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 12732
I20260807 15:10:36.447790 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:36.447963 18948 external_mini_cluster.cc:1428] Setting key 480db5c3201f7e5815a71e44ff9f7988
Aug 07 15:10:36 dist-test-slave-5bn2 krb5kdc[12367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:36.454759 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.445782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:33551 (local address 127.18.129.62:34337)
0807 15:10:36.445864 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.445867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.446030 (+ 163us) server_negotiation.cc:407] Connection header received
0807 15:10:36.446488 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.446489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.446499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.446551 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:36.447533 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.448127 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.449416 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.449603 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.449650 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.452036 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:36.452038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:36.452038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:36.452045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:36.453345 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.453821 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.453823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.453823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.453888 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.454111 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.454112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.454113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.454253 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:36.454319 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.454568 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.454664 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:10:36.454843 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.445743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34337 (local address 127.18.129.2:33551)
0807 15:10:36.445894 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.445932 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.446412 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.446609 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.446613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.447243 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:36.447445 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.447450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.448227 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.448228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.449327 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.449330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.449678 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.449680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.449819 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.450299 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:36.450310 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:36.451952 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:36.453436 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:36.453444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:36.453446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:36.453750 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:36.453953 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:36.453954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:36.453954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:36.454055 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:36.454316 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:36.454319 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:36.454427 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.454498 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.454655 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260807 15:10:36.455294 12921 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34337
I20260807 15:10:36.455399 12921 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:36.455610 12921 heartbeater.cc:511] Master 127.18.129.62:34337 requested a full tablet report, sending...
I20260807 15:10:36.455998 12437 ts_manager.cc:194] Registered new tserver with Master: 71cb979e6fa0407c9bfd549bdad549af (127.18.129.2:37295)
I20260807 15:10:36.456351 12437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:33551
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:10:36.475399 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34337
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:36.556604 12926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:36.556815 12926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:36.556838 12926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:36.557992 12926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:36.558036 12926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:36.558064 12926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:10:36.559556 12926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:36.559782 12926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:36.559998 12926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:36.565177 12933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:36.565165 12937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:36.565165 12935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:36.565447 12926 server_base.cc:1034] running on GCE node
I20260807 15:10:36.565676 12926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:36.565901 12926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:36.567059 12926 hybrid_clock.cc:648] HybridClock initialized: now 1786115436567042 us; error 28 us; skew 500 ppm
Aug 07 15:10:36 dist-test-slave-5bn2 krb5kdc[12367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:36.569849 12926 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:10:36.570327 12926 webserver.cc:460] Webserver started at http://127.18.129.3:43347/ using document root <none> and password file <none>
I20260807 15:10:36.570549 12926 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:36.570590 12926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:36.570684 12926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:36.571817 12926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "793fca0c77f3423eaf8b1f488f8dae46"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "a07dca6fe54680ec37c0eb12386e9bb7"
server_key_iv: "6982a582c877091fa0d20c0d4ff65d37"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.572134 12926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "793fca0c77f3423eaf8b1f488f8dae46"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "a07dca6fe54680ec37c0eb12386e9bb7"
server_key_iv: "6982a582c877091fa0d20c0d4ff65d37"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.573482 12926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:36.574209 12944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.574468 12926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:36.574549 12926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "793fca0c77f3423eaf8b1f488f8dae46"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "a07dca6fe54680ec37c0eb12386e9bb7"
server_key_iv: "6982a582c877091fa0d20c0d4ff65d37"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.574625 12926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:36.593389 12926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:36.595942 12926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:36.596102 12926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:36.596383 12926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:36.596731 12926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:36.596771 12926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.596798 12926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:36.596827 12926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.603813 12926 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:34877
I20260807 15:10:36.603854 13120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:34877 every 8 connection(s)
I20260807 15:10:36.604200 12926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:36.610519 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 12926
I20260807 15:10:36.610662 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:36.610865 18948 external_mini_cluster.cc:1428] Setting key 8a57e045cf6caac61deac1381244b19d
Aug 07 15:10:36 dist-test-slave-5bn2 krb5kdc[12367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:36.613893 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.604920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:40285 (local address 127.18.129.62:34337)
0807 15:10:36.604970 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.604972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.605266 (+ 294us) server_negotiation.cc:407] Connection header received
0807 15:10:36.605789 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.605791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.605802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.605887 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:36.606945 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.607499 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.608799 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.609002 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.609071 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.611509 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:36.611510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:36.611511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:36.611518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:36.612616 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.612992 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.612993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.612993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.613047 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.613275 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.613276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.613276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.613402 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:36.613460 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.613724 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.613804 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:10:36.614068 13123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.604891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34337 (local address 127.18.129.3:40285)
0807 15:10:36.605100 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.605147 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.605670 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.606006 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.606010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.606625 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:36.606825 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.606830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.607620 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.607621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.608719 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.608723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.609116 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.609117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.609265 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.609778 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:36.609789 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:36.611417 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:36.612694 (+ 1277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:36.612701 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:36.612703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:36.612922 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:36.613108 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:36.613108 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:36.613109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:36.613215 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:36.613465 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:36.613472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:36.613605 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.613693 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.613846 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260807 15:10:36.614476 13121 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34337
I20260807 15:10:36.614573 13121 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:36.614748 13121 heartbeater.cc:511] Master 127.18.129.62:34337 requested a full tablet report, sending...
I20260807 15:10:36.615131 12437 ts_manager.cc:194] Registered new tserver with Master: 793fca0c77f3423eaf8b1f488f8dae46 (127.18.129.3:34877)
I20260807 15:10:36.615523 12437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:40285
I20260807 15:10:36.622947 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:10:36.629961 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.624271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54520 (local address 127.18.129.62:34337)
0807 15:10:36.624314 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.624315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.624398 (+ 83us) server_negotiation.cc:407] Connection header received
0807 15:10:36.624437 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.624438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.624447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.624491 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:36.625093 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.625572 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.626255 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.626432 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.626477 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.627678 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:36.627679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:36.627679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:36.627685 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:36.629049 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.629345 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.629346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.629346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.629395 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:36.629529 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:36.629529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:36.629530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:36.629674 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:36.629748 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.629818 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.629889 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:10:36.631623 12437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:36.633145 12437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:36.639561 13140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.635582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53364 (local address 127.18.129.1:35539)
0807 15:10:36.635990 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.635995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.636004 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:10:36.636045 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.636048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.636263 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.636463 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.636536 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.637581 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.638589 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.639183 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.639247 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.639306 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.639441 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.639482 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":79,"threads_started":1}
I20260807 15:10:36.639562 13139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.635424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35539 (local address 127.0.0.1:53364)
0807 15:10:36.635571 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.635613 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.635674 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.636361 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.636361 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.636364 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.636400 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.636401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.637673 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.637674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.638507 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.638510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.639262 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.639263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.639341 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.639354 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.639419 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.639475 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":46,"threads_started":1}
I20260807 15:10:36.639846 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.635431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:37295 (local address 127.0.0.1:52444)
0807 15:10:36.635773 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.635782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.635815 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.636141 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.636144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.636146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.636215 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.636217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.637561 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.637562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.638610 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.638613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.639474 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.639475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.639542 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.639544 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.639571 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.639748 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":8,"thread_start_us":85,"threads_started":1}
I20260807 15:10:36.639919 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.635770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49348 (local address 127.18.129.3:34877)
0807 15:10:36.635970 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.635974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.636072 (+ 98us) server_negotiation.cc:407] Connection header received
0807 15:10:36.636113 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.636115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.636129 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.636194 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.636394 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.637087 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.638468 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.639256 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.639328 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.639384 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.639621 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.639846 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260807 15:10:36.639829 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.635645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:34877 (local address 127.0.0.1:49348)
0807 15:10:36.635961 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.635972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.636036 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.636277 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.636278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.636280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.636334 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.636336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.637183 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.637184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.638378 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.638381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.639422 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.639424 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.639488 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.639492 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.639534 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.639573 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":84,"threads_started":1}
I20260807 15:10:36.640380 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.635784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52444 (local address 127.18.129.2:37295)
0807 15:10:36.635950 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.635955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.635967 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:10:36.636029 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.636032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.636049 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.636128 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.636305 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.637446 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.638676 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.639391 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.640156 (+ 765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.640232 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.640258 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.640304 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260807 15:10:36.640322 12619 tablet_service.cc:1467] Processing CreateTablet for tablet 48908d794b09438db5572e9d5ec27b58 (DEFAULT_TABLE table=test-table [id=99cb93a7081347d88c6e9e440221d0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:36.640722 12619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48908d794b09438db5572e9d5ec27b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:36.640717 13024 tablet_service.cc:1467] Processing CreateTablet for tablet 48908d794b09438db5572e9d5ec27b58 (DEFAULT_TABLE table=test-table [id=99cb93a7081347d88c6e9e440221d0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:36.641053 13024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48908d794b09438db5572e9d5ec27b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:36.641052 12823 tablet_service.cc:1467] Processing CreateTablet for tablet 48908d794b09438db5572e9d5ec27b58 (DEFAULT_TABLE table=test-table [id=99cb93a7081347d88c6e9e440221d0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:36.641409 12823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48908d794b09438db5572e9d5ec27b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:36.644140 13145 tablet_bootstrap.cc:492] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Bootstrap starting.
I20260807 15:10:36.644433 13148 tablet_bootstrap.cc:492] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: Bootstrap starting.
I20260807 15:10:36.644814 13145 tablet_bootstrap.cc:654] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:36.644976 13148 tablet_bootstrap.cc:654] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:36.645284 13145 log.cc:824] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:36.645460 13148 log.cc:824] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:36.645810 13147 tablet_bootstrap.cc:492] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692: Bootstrap starting.
I20260807 15:10:36.646062 13145 tablet_bootstrap.cc:492] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: No bootstrap required, opened a new log
I20260807 15:10:36.646124 13145 ts_tablet_manager.cc:1397] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:36.646358 13147 tablet_bootstrap.cc:654] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:36.646804 13147 log.cc:824] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:36.647258 13148 tablet_bootstrap.cc:492] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: No bootstrap required, opened a new log
I20260807 15:10:36.647372 13148 ts_tablet_manager.cc:1397] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:36.647553 13147 tablet_bootstrap.cc:492] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692: No bootstrap required, opened a new log
I20260807 15:10:36.647624 13147 ts_tablet_manager.cc:1397] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:36.648084 13145 raft_consensus.cc:348] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.648279 13145 raft_consensus.cc:374] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:36.648319 13145 raft_consensus.cc:729] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71cb979e6fa0407c9bfd549bdad549af, State: Initialized, Role: FOLLOWER
I20260807 15:10:36.648459 13145 consensus_queue.cc:260] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.648770 12921 heartbeater.cc:503] Master 127.18.129.62:34337 was elected leader, sending a full tablet report...
I20260807 15:10:36.648758 13145 ts_tablet_manager.cc:1428] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:36.649302 13148 raft_consensus.cc:348] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.649286 13147 raft_consensus.cc:348] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.649423 13148 raft_consensus.cc:374] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:36.649446 13147 raft_consensus.cc:374] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:36.649454 13148 raft_consensus.cc:729] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 793fca0c77f3423eaf8b1f488f8dae46, State: Initialized, Role: FOLLOWER
I20260807 15:10:36.649480 13147 raft_consensus.cc:729] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 070a905fe8fe458b8793076182fac692, State: Initialized, Role: FOLLOWER
I20260807 15:10:36.649537 13148 consensus_queue.cc:260] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.649612 13147 consensus_queue.cc:260] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.649798 13121 heartbeater.cc:503] Master 127.18.129.62:34337 was elected leader, sending a full tablet report...
I20260807 15:10:36.649807 13148 ts_tablet_manager.cc:1428] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:36.649878 13147 ts_tablet_manager.cc:1428] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:36.650035 12722 heartbeater.cc:503] Master 127.18.129.62:34337 was elected leader, sending a full tablet report...
W20260807 15:10:36.695662 12922 tablet.cc:2369] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:36.764954 13152 raft_consensus.cc:482] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:36.765082 13152 raft_consensus.cc:504] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.765448 13152 leader_election.cc:288] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 070a905fe8fe458b8793076182fac692 (127.18.129.1:35539), 793fca0c77f3423eaf8b1f488f8dae46 (127.18.129.3:34877)
I20260807 15:10:36.767154 13153 raft_consensus.cc:482] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:36.767280 13153 raft_consensus.cc:504] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.767627 13153 leader_election.cc:288] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 070a905fe8fe458b8793076182fac692 (127.18.129.1:35539), 71cb979e6fa0407c9bfd549bdad549af (127.18.129.2:37295)
I20260807 15:10:36.768780 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.765626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35539 (local address 127.18.129.2:53469)
0807 15:10:36.765686 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.765699 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.765757 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.765915 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.765916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.765918 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.765999 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.766001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.766783 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.766783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.767788 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.767791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.768542 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.768543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.768619 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.768623 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.768665 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.768704 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 15:10:36.768834 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.765835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:33089 (local address 127.18.129.3:34877)
0807 15:10:36.765906 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.765908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.765993 (+ 85us) server_negotiation.cc:407] Connection header received
0807 15:10:36.766033 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.766034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.766044 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.766095 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.766261 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.766720 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.767945 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.768463 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.768503 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.768512 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.768732 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.768769 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:10:36.769047 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.765748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:34877 (local address 127.18.129.2:33089)
0807 15:10:36.765929 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.765938 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.765991 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.766118 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.766119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.766120 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.766199 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.766200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.766801 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.766802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.767842 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.767846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.768568 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.768569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.768651 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.768655 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.768931 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.768972 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":116,"threads_started":1}
I20260807 15:10:36.769049 13140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.765704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:53469 (local address 127.18.129.1:35539)
0807 15:10:36.765780 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.765782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.765793 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:10:36.765824 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.765825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.765835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.765905 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.766135 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.766693 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.767920 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.768447 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.768505 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.768513 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.768942 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.768980 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:10:36.769318 12650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48908d794b09438db5572e9d5ec27b58" candidate_uuid: "71cb979e6fa0407c9bfd549bdad549af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "070a905fe8fe458b8793076182fac692" is_pre_election: true
I20260807 15:10:36.769446 12650 raft_consensus.cc:2454] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71cb979e6fa0407c9bfd549bdad549af in term 0.
I20260807 15:10:36.769711 12761 leader_election.cc:302] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 070a905fe8fe458b8793076182fac692, 71cb979e6fa0407c9bfd549bdad549af; no voters:
I20260807 15:10:36.769759 13054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48908d794b09438db5572e9d5ec27b58" candidate_uuid: "71cb979e6fa0407c9bfd549bdad549af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "793fca0c77f3423eaf8b1f488f8dae46" is_pre_election: true
I20260807 15:10:36.769838 13152 raft_consensus.cc:2781] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:36.769860 13054 raft_consensus.cc:2454] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71cb979e6fa0407c9bfd549bdad549af in term 0.
I20260807 15:10:36.769886 13152 raft_consensus.cc:482] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:36.769908 13152 raft_consensus.cc:3037] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:36.770778 13152 raft_consensus.cc:504] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.770910 13152 leader_election.cc:288] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [CANDIDATE]: Term 1 election: Requested vote from peers 070a905fe8fe458b8793076182fac692 (127.18.129.1:35539), 793fca0c77f3423eaf8b1f488f8dae46 (127.18.129.3:34877)
I20260807 15:10:36.771138 13054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48908d794b09438db5572e9d5ec27b58" candidate_uuid: "71cb979e6fa0407c9bfd549bdad549af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "793fca0c77f3423eaf8b1f488f8dae46"
I20260807 15:10:36.771150 12650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48908d794b09438db5572e9d5ec27b58" candidate_uuid: "71cb979e6fa0407c9bfd549bdad549af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "070a905fe8fe458b8793076182fac692"
I20260807 15:10:36.771217 13054 raft_consensus.cc:3037] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:36.771226 12650 raft_consensus.cc:3037] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:36.771334 13123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.767749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35539 (local address 127.18.129.3:53099)
0807 15:10:36.767819 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.767828 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.767860 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.768392 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.768393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.768395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.768445 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.768447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.769287 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.769288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.770386 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.770388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.771120 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.771121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.771196 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.771200 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.771233 (+ 33us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.771277 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":16}
I20260807 15:10:36.771445 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.768124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:58839 (local address 127.18.129.2:37295)
0807 15:10:36.768167 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.768168 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.768175 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:10:36.768192 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.768192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.768200 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.768240 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.768427 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.769043 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.770587 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.771075 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.771146 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.771154 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.771359 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.771395 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:10:36.771605 13162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.767992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:53099 (local address 127.18.129.1:35539)
0807 15:10:36.768203 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.768205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.768212 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:10:36.768234 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.768234 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.768242 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.768506 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.768541 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.769205 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.770502 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.771016 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.771427 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.771437 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.771455 (+ 18us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.771538 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":79,"threads_started":1}
I20260807 15:10:36.771638 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.767810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:37295 (local address 127.18.129.3:58839)
0807 15:10:36.768006 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.768016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.768057 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.768295 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.768295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.768297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.768358 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.768360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.769163 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.769164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.770512 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.770514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.771202 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.771203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.771274 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.771278 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.771527 (+ 249us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.771571 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260807 15:10:36.772119 12650 raft_consensus.cc:2454] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71cb979e6fa0407c9bfd549bdad549af in term 1.
I20260807 15:10:36.772130 13054 raft_consensus.cc:2454] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71cb979e6fa0407c9bfd549bdad549af in term 1.
I20260807 15:10:36.772141 12854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48908d794b09438db5572e9d5ec27b58" candidate_uuid: "793fca0c77f3423eaf8b1f488f8dae46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71cb979e6fa0407c9bfd549bdad549af" is_pre_election: true
I20260807 15:10:36.772241 12854 raft_consensus.cc:2379] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 793fca0c77f3423eaf8b1f488f8dae46 in current term 1: Already voted for candidate 71cb979e6fa0407c9bfd549bdad549af in this term.
I20260807 15:10:36.772284 12761 leader_election.cc:302] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 070a905fe8fe458b8793076182fac692, 71cb979e6fa0407c9bfd549bdad549af; no voters:
I20260807 15:10:36.772433 13152 raft_consensus.cc:2781] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:36.772466 12650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48908d794b09438db5572e9d5ec27b58" candidate_uuid: "793fca0c77f3423eaf8b1f488f8dae46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "070a905fe8fe458b8793076182fac692" is_pre_election: true
I20260807 15:10:36.772545 12650 raft_consensus.cc:2379] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 793fca0c77f3423eaf8b1f488f8dae46 in current term 1: Already voted for candidate 71cb979e6fa0407c9bfd549bdad549af in this term.
I20260807 15:10:36.772605 13152 raft_consensus.cc:686] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 1 LEADER]: Becoming Leader. State: Replica: 71cb979e6fa0407c9bfd549bdad549af, State: Running, Role: LEADER
I20260807 15:10:36.772701 13152 consensus_queue.cc:237] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } }
I20260807 15:10:36.772783 12949 leader_election.cc:302] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 793fca0c77f3423eaf8b1f488f8dae46; no voters: 070a905fe8fe458b8793076182fac692, 71cb979e6fa0407c9bfd549bdad549af
I20260807 15:10:36.772900 13153 raft_consensus.cc:2726] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 15:10:36.773614 12436 catalog_manager.cc:5495] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af reported cstate change: term changed from 0 to 1, leader changed from <none> to 71cb979e6fa0407c9bfd549bdad549af (127.18.129.2). New cstate: current_term: 1 leader_uuid: "71cb979e6fa0407c9bfd549bdad549af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71cb979e6fa0407c9bfd549bdad549af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260807 15:10:36.782791 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.780817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.18.129.2:37295)
0807 15:10:36.780859 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.780860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.780867 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:10:36.780931 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.780932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.780947 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.781002 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:36.781136 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.781559 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.782249 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.782413 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.782454 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.782560 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:36.782653 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:36.782703 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.782718 (+ 15us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.782731 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:10:36.784423 12650 raft_consensus.cc:1261] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 1 FOLLOWER]: Refusing update from remote peer 71cb979e6fa0407c9bfd549bdad549af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:36.784427 13054 raft_consensus.cc:1261] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 1 FOLLOWER]: Refusing update from remote peer 71cb979e6fa0407c9bfd549bdad549af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:36.784667 13152 consensus_queue.cc:1035] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [LEADER]: Connected to new peer: Peer: permanent_uuid: "070a905fe8fe458b8793076182fac692" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:36.784809 13163 consensus_queue.cc:1035] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [LEADER]: Connected to new peer: Peer: permanent_uuid: "793fca0c77f3423eaf8b1f488f8dae46" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:36.790011 12436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:36.790127 12436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:36.790910 12436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bd9e07ab8904fb0bc14d2716d421100: Sending DeleteTablet for 3 replicas of tablet 48908d794b09438db5572e9d5ec27b58
I20260807 15:10:36.791285 12823 tablet_service.cc:1514] Processing DeleteTablet for tablet 48908d794b09438db5572e9d5ec27b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:36 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:52444
I20260807 15:10:36.791433 12619 tablet_service.cc:1514] Processing DeleteTablet for tablet 48908d794b09438db5572e9d5ec27b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:36 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:53364
I20260807 15:10:36.791574 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 12530
I20260807 15:10:36.791692 13177 tablet_replica.cc:330] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692: stopping tablet replica
I20260807 15:10:36.791678 13024 tablet_service.cc:1514] Processing DeleteTablet for tablet 48908d794b09438db5572e9d5ec27b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:36 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:49348
I20260807 15:10:36.791801 13177 raft_consensus.cc:2229] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:36.791894 13177 raft_consensus.cc:2258] T 48908d794b09438db5572e9d5ec27b58 P 070a905fe8fe458b8793076182fac692 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:36.802230 13179 tablet_replica.cc:330] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: stopping tablet replica
I20260807 15:10:36.802382 13179 raft_consensus.cc:2229] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:36.802500 13179 raft_consensus.cc:2258] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:36.803031 13179 ts_tablet_manager.cc:1905] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:36.802713 13176 tablet_replica.cc:330] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: stopping tablet replica
I20260807 15:10:36.804742 13179 ts_tablet_manager.cc:1918] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:36.804817 13179 log.cc:1196] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/48908d794b09438db5572e9d5ec27b58
I20260807 15:10:36.805101 13179 ts_tablet_manager.cc:1939] T 48908d794b09438db5572e9d5ec27b58 P 793fca0c77f3423eaf8b1f488f8dae46: Deleting consensus metadata
I20260807 15:10:36.805485 13176 raft_consensus.cc:2229] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:36.805616 13176 raft_consensus.cc:2258] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:36.805624 12416 catalog_manager.cc:4841] TS 793fca0c77f3423eaf8b1f488f8dae46 (127.18.129.3:34877): tablet 48908d794b09438db5572e9d5ec27b58 (table test-table [id=99cb93a7081347d88c6e9e440221d0fa]) successfully deleted
I20260807 15:10:36.806032 13176 ts_tablet_manager.cc:1905] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:36.807549 13176 ts_tablet_manager.cc:1918] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:36.807629 13176 log.cc:1196] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/48908d794b09438db5572e9d5ec27b58
I20260807 15:10:36.807889 13176 ts_tablet_manager.cc:1939] T 48908d794b09438db5572e9d5ec27b58 P 71cb979e6fa0407c9bfd549bdad549af: Deleting consensus metadata
W20260807 15:10:36.808223 12416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:35539 (error 108)
I20260807 15:10:36.808297 12410 catalog_manager.cc:4841] TS 71cb979e6fa0407c9bfd549bdad549af (127.18.129.2:37295): tablet 48908d794b09438db5572e9d5ec27b58 (table test-table [id=99cb93a7081347d88c6e9e440221d0fa]) successfully deleted
I20260807 15:10:36.808449 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 12732
I20260807 15:10:36.813795 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.813635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:35539 (local address 127.0.0.1:53368)
0807 15:10:36.813709 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.813760 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:35539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 15:10:36.814580 12416 catalog_manager.cc:4584] TS 070a905fe8fe458b8793076182fac692 (127.18.129.1:35539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48908d794b09438db5572e9d5ec27b58: Network error: Client connection negotiation failed: client connection to 127.18.129.1:35539: connect: Connection refused (error 111)
I20260807 15:10:36.822203 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 12926
I20260807 15:10:36.833703 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 12388
2026-08-07T15:10:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2133 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T15:10:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:36Z Disabled control of system clock
I20260807 15:10:36.865991 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34065
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:36905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:34065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:36.946205 13185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:36.946431 13185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:36.946449 13185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:36.947618 13185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:36.947660 13185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:36.947671 13185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:36.947683 13185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:36.947695 13185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:36.949033 13185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:34065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:34065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:36.949271 13185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:36.949550 13185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:36.954492 13197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:36.954489 13200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:36.954484 13196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:36.954655 13185 server_base.cc:1034] running on GCE node
I20260807 15:10:36.954970 13185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:36.955250 13185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:36.956390 13185 hybrid_clock.cc:648] HybridClock initialized: now 1786115436956375 us; error 28 us; skew 500 ppm
I20260807 15:10:36.957603 13185 webserver.cc:460] Webserver started at http://127.18.129.62:33679/ using document root <none> and password file <none>
I20260807 15:10:36.957831 13185 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:36.957880 13185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:36.958012 13185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:36.959141 13185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "97beb77c269d4eb5a65bc9d1408d5f7c"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "5dde5e711927ec8d66ebeccf844bb0a7"
server_key_iv: "f283eb53870fccf01584490fd28866f9"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.959473 13185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "97beb77c269d4eb5a65bc9d1408d5f7c"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "5dde5e711927ec8d66ebeccf844bb0a7"
server_key_iv: "f283eb53870fccf01584490fd28866f9"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.960645 13185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:36.961303 13210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:36.961465 13185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:36.961550 13185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "97beb77c269d4eb5a65bc9d1408d5f7c"
format_stamp: "Formatted at 2026-08-07 15:10:36 on dist-test-slave-5bn2"
server_key: "5dde5e711927ec8d66ebeccf844bb0a7"
server_key_iv: "f283eb53870fccf01584490fd28866f9"
server_key_version: "encryptionkey@0"
I20260807 15:10:36.961630 13185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:36.981487 13185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:36.981793 13185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:36.981930 13185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:36.985690 13185 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:34065
I20260807 15:10:36.985724 13288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:34065 every 8 connection(s)
I20260807 15:10:36.986034 13185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:36.987043 13290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:36.989315 13290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c: Bootstrap starting.
I20260807 15:10:36.989873 13290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:36.990163 13290 log.cc:824] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:36.990531 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 13185
I20260807 15:10:36.990662 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:36.990893 18948 external_mini_cluster.cc:1428] Setting key 77f4745b330dc6a74cc1c6e5ae619a8d
I20260807 15:10:36.990911 13290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c: No bootstrap required, opened a new log
I20260807 15:10:36.993348 13290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97beb77c269d4eb5a65bc9d1408d5f7c" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34065 } }
I20260807 15:10:36.993525 13290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:36.993573 13290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97beb77c269d4eb5a65bc9d1408d5f7c, State: Initialized, Role: FOLLOWER
I20260807 15:10:36.993692 13290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97beb77c269d4eb5a65bc9d1408d5f7c" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34065 } }
I20260807 15:10:36.993772 13290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:36.993803 13290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:36.993840 13290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:36.994481 13290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97beb77c269d4eb5a65bc9d1408d5f7c" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34065 } }
I20260807 15:10:36.994580 13290 leader_election.cc:302] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97beb77c269d4eb5a65bc9d1408d5f7c; no voters:
I20260807 15:10:36.994727 13290 leader_election.cc:288] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:36.994781 13296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:36.994946 13296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [term 1 LEADER]: Becoming Leader. State: Replica: 97beb77c269d4eb5a65bc9d1408d5f7c, State: Running, Role: LEADER
I20260807 15:10:36.994992 13290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:36.995046 13296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97beb77c269d4eb5a65bc9d1408d5f7c" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34065 } }
I20260807 15:10:36.995389 13296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97beb77c269d4eb5a65bc9d1408d5f7c. Latest consensus state: current_term: 1 leader_uuid: "97beb77c269d4eb5a65bc9d1408d5f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97beb77c269d4eb5a65bc9d1408d5f7c" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34065 } } }
I20260807 15:10:36.995460 13296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:36.995777 13298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97beb77c269d4eb5a65bc9d1408d5f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97beb77c269d4eb5a65bc9d1408d5f7c" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 34065 } } }
I20260807 15:10:36.995904 13298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:36.996575 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.991409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.18.129.62:34065)
0807 15:10:36.991613 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.991621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.991646 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:10:36.992101 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.992108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.992162 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.992271 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:36.992486 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.993340 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.994409 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.994966 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.995432 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.995835 (+ 403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:36.995839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:36.995839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:36.996039 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:36.996081 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:36.996123 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:36.996187 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.996301 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.996387 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
W20260807 15:10:36.996850 13319 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:10:36.996901 13319 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:10:36.996960 13306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:36.997129 13306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:36.998543 13306 catalog_manager.cc:1347] Generated new cluster ID: 543f092808e245d09e1f40df0524fb2d
I20260807 15:10:36.998602 13306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:37.013391 13306 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:37.014119 13306 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:37.022598 13306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c: Generated new TSK 0
I20260807 15:10:37.022778 13306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:37.030069 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34065
--builtin_ntp_servers=127.18.129.20:36905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:37.114801 13325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:37.115095 13325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:37.115128 13325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:37.116376 13325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:37.116420 13325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:37.116443 13325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:10:37.117882 13325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:37.118156 13325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:37.118494 13325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:37.124089 13337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:37.124205 13334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:37.124104 13335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:37.124106 13325 server_base.cc:1034] running on GCE node
I20260807 15:10:37.124598 13325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:37.124841 13325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:37.126016 13325 hybrid_clock.cc:648] HybridClock initialized: now 1786115437125989 us; error 43 us; skew 500 ppm
I20260807 15:10:37.127382 13325 webserver.cc:460] Webserver started at http://127.18.129.1:33495/ using document root <none> and password file <none>
I20260807 15:10:37.127667 13325 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:37.127724 13325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:37.127853 13325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:37.128975 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "54e6415196984c34bd92932cc57639f2"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "b787ad674f05cca98d5a40446b01fef2"
server_key_iv: "2507bf896ec74e17bd925fd47f3277a5"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.129314 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "54e6415196984c34bd92932cc57639f2"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "b787ad674f05cca98d5a40446b01fef2"
server_key_iv: "2507bf896ec74e17bd925fd47f3277a5"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.130641 13325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:10:37.131433 13345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.131618 13325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.131712 13325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "54e6415196984c34bd92932cc57639f2"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "b787ad674f05cca98d5a40446b01fef2"
server_key_iv: "2507bf896ec74e17bd925fd47f3277a5"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.131788 13325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:37.152169 13325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:37.152477 13325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:37.152582 13325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:37.152776 13325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:37.153075 13325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:37.153106 13325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.153129 13325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:37.153146 13325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.158749 13325 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:37711
I20260807 15:10:37.158831 13522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:37711 every 8 connection(s)
I20260807 15:10:37.159161 13325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:37.164215 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.159902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:60999 (local address 127.18.129.62:34065)
0807 15:10:37.159968 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.159970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.160198 (+ 228us) server_negotiation.cc:407] Connection header received
0807 15:10:37.160690 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.160691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.160705 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.160765 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:37.161412 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.161906 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.162874 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.163094 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.163148 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.163907 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:37.163909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:37.163909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:37.163913 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:37.163929 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:37.163933 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:37.163978 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.164081 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.164127 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:10:37.164418 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.159876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34065 (local address 127.18.129.1:60999)
0807 15:10:37.160078 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.160120 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:37.160611 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:37.160820 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:37.160823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:37.160831 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:37.161343 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.161347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.161979 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.161979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.162780 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.162783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.163166 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.163167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.163282 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.163772 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:37.163772 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:37.163801 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:37.163828 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:37.164021 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:37.164025 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:37.164027 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:10:37.164077 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:10:37.164205 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260807 15:10:37.164913 13525 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34065
I20260807 15:10:37.165012 13525 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:37.165189 13525 heartbeater.cc:511] Master 127.18.129.62:34065 requested a full tablet report, sending...
I20260807 15:10:37.165742 13237 ts_manager.cc:194] Registered new tserver with Master: 54e6415196984c34bd92932cc57639f2 (127.18.129.1:37711)
I20260807 15:10:37.165815 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 13325
I20260807 15:10:37.165922 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:37.166149 18948 external_mini_cluster.cc:1428] Setting key 9dad874d652fe683a7706a6e412bd4d8
I20260807 15:10:37.166745 13237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.1:60999
I20260807 15:10:37.167294 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34065
--builtin_ntp_servers=127.18.129.20:36905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:37.250829 13528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:37.251080 13528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:37.251102 13528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:37.252224 13528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:37.252262 13528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:37.252286 13528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:10:37.253624 13528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:37.253822 13528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:37.254071 13528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:37.259285 13543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:37.259341 13528 server_base.cc:1034] running on GCE node
W20260807 15:10:37.259274 13539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:37.259294 13538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:37.259696 13528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:37.259925 13528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:37.261068 13528 hybrid_clock.cc:648] HybridClock initialized: now 1786115437261047 us; error 35 us; skew 500 ppm
I20260807 15:10:37.262375 13528 webserver.cc:460] Webserver started at http://127.18.129.2:35277/ using document root <none> and password file <none>
I20260807 15:10:37.262598 13528 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:37.262648 13528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:37.262780 13528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:37.263912 13528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "cc1db3f6d10a469f81b87c77580f5ec0"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "6cfc04eda2241f2dde8c25dfcf97c0ec"
server_key_iv: "21b68a606a22c80fff9df661e92247d0"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.264288 13528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "cc1db3f6d10a469f81b87c77580f5ec0"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "6cfc04eda2241f2dde8c25dfcf97c0ec"
server_key_iv: "21b68a606a22c80fff9df661e92247d0"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.265597 13528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:37.266307 13554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.266502 13528 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:37.266590 13528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "cc1db3f6d10a469f81b87c77580f5ec0"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "6cfc04eda2241f2dde8c25dfcf97c0ec"
server_key_iv: "21b68a606a22c80fff9df661e92247d0"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.266657 13528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:37.287009 13528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:37.287360 13528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:37.287515 13528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:37.287758 13528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:37.288128 13528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:37.288170 13528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.288203 13528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:37.288226 13528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.293926 13528 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:38351
I20260807 15:10:37.293977 13727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:38351 every 8 connection(s)
I20260807 15:10:37.294333 13528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:37.299325 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.295061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:45693 (local address 127.18.129.62:34065)
0807 15:10:37.295153 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.295155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.295357 (+ 202us) server_negotiation.cc:407] Connection header received
0807 15:10:37.295896 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.295897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.295906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.295965 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:37.296620 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.297170 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.298099 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.298272 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.298315 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.299122 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:37.299123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:37.299123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:37.299127 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:37.299140 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:37.299143 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:37.299183 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.299245 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.299265 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:10:37.299530 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.295034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34065 (local address 127.18.129.2:45693)
0807 15:10:37.295249 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.295290 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:37.295813 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:37.296027 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:37.296031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:37.296038 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:37.296551 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.296556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.297250 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.297251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.298025 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.298027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.298328 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.298329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.298431 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.298980 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:37.298980 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:37.299024 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:37.299048 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:37.299193 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:37.299195 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:37.299197 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:10:37.299242 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:10:37.299323 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":85,"threads_started":1}
I20260807 15:10:37.299971 13728 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34065
I20260807 15:10:37.300067 13728 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:37.300213 13728 heartbeater.cc:511] Master 127.18.129.62:34065 requested a full tablet report, sending...
I20260807 15:10:37.300555 13237 ts_manager.cc:194] Registered new tserver with Master: cc1db3f6d10a469f81b87c77580f5ec0 (127.18.129.2:38351)
I20260807 15:10:37.300901 13237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.2:45693
I20260807 15:10:37.302838 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 13528
I20260807 15:10:37.302960 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:37.303174 18948 external_mini_cluster.cc:1428] Setting key 46d62ec7880e3507f4a60ff5e5bdeac6
I20260807 15:10:37.304021 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34065
--builtin_ntp_servers=127.18.129.20:36905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:37.385917 13731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:37.386121 13731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:37.386147 13731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:37.387288 13731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:37.387331 13731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:37.387363 13731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:10:37.388708 13731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:34065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:37.388947 13731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:37.389241 13731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:37.394579 13742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:37.394688 13731 server_base.cc:1034] running on GCE node
W20260807 15:10:37.394589 13738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:37.394587 13745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:37.395108 13731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:37.395347 13731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:37.396530 13731 hybrid_clock.cc:648] HybridClock initialized: now 1786115437396514 us; error 28 us; skew 500 ppm
I20260807 15:10:37.397801 13731 webserver.cc:460] Webserver started at http://127.18.129.3:39093/ using document root <none> and password file <none>
I20260807 15:10:37.398072 13731 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:37.398126 13731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:37.398258 13731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:37.399397 13731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "260e31f0493b4faf8348b1b5ef2dea87"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "e39f90dee5d3760f83c9b425fa1b50ae"
server_key_iv: "865930d8ab5582d3798bd7351a92f0ff"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.399746 13731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "260e31f0493b4faf8348b1b5ef2dea87"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "e39f90dee5d3760f83c9b425fa1b50ae"
server_key_iv: "865930d8ab5582d3798bd7351a92f0ff"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.400964 13731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:37.401612 13754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.401765 13731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:37.401858 13731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "260e31f0493b4faf8348b1b5ef2dea87"
format_stamp: "Formatted at 2026-08-07 15:10:37 on dist-test-slave-5bn2"
server_key: "e39f90dee5d3760f83c9b425fa1b50ae"
server_key_iv: "865930d8ab5582d3798bd7351a92f0ff"
server_key_version: "encryptionkey@0"
I20260807 15:10:37.401930 13731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:37.419214 13731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:37.419499 13731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:37.419660 13731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:37.419884 13731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:37.420207 13731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:37.420243 13731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.420272 13731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:37.420289 13731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:37.425925 13731 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:40469
I20260807 15:10:37.425980 13915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:40469 every 8 connection(s)
I20260807 15:10:37.426297 13731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:37.429410 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 13731
I20260807 15:10:37.429513 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:37.429692 18948 external_mini_cluster.cc:1428] Setting key c9b5baf4cff95c25a9e39e0fd0317a84
I20260807 15:10:37.431682 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.427052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:38657 (local address 127.18.129.62:34065)
0807 15:10:37.427127 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.427129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.427360 (+ 231us) server_negotiation.cc:407] Connection header received
0807 15:10:37.427834 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.427835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.427845 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.427893 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:37.428551 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.429023 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.430242 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.430528 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.430589 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.431460 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:37.431462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:37.431462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:37.431466 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:37.431483 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:37.431486 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:37.431538 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.431597 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.431622 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:10:37.431876 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.427057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:34065 (local address 127.18.129.3:38657)
0807 15:10:37.427233 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.427269 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:37.427755 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:37.427955 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:37.427958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:37.427966 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:37.428482 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.428487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.429124 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.429125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.430075 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.430078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.430612 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.430613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.430775 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.431279 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:37.431279 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:37.431309 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:37.431333 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:37.431544 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:37.431546 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:37.431549 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:37.431592 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:10:37.431670 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260807 15:10:37.432341 13916 heartbeater.cc:347] Connected to a master server at 127.18.129.62:34065
I20260807 15:10:37.432420 13916 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:37.432610 13916 heartbeater.cc:511] Master 127.18.129.62:34065 requested a full tablet report, sending...
I20260807 15:10:37.432993 13237 ts_manager.cc:194] Registered new tserver with Master: 260e31f0493b4faf8348b1b5ef2dea87 (127.18.129.3:40469)
I20260807 15:10:37.433333 13237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.3:38657
I20260807 15:10:37.441395 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:10:37.445227 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.442708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52688 (local address 127.18.129.62:34065)
0807 15:10:37.442776 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.442777 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.442869 (+ 92us) server_negotiation.cc:407] Connection header received
0807 15:10:37.442950 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.442951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.442960 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.443006 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:37.443144 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.443610 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.444281 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.444453 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.444494 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.445056 (+ 562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:37.445058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:37.445058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:37.445062 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:37.445077 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:37.445080 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:37.445117 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.445150 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.445170 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:10:37.446727 13237 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:37.448276 13237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:37.454453 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.450508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:37711 (local address 127.0.0.1:52048)
0807 15:10:37.450639 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.450649 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:37.450694 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:37.451034 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:37.451037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:37.451039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:37.451154 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.451157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.452098 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.452099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.453207 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.453210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.454109 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.454110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.454198 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.454214 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 15:10:37.454295 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:10:37.454367 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":28,"thread_start_us":76,"threads_started":1}
I20260807 15:10:37.454594 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.450402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.18.129.2:38351)
0807 15:10:37.450843 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.450866 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.450876 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:10:37.450940 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.450942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.450956 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.451216 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:37.451282 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.452416 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.453650 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.454220 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.454303 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.454392 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.454465 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.454507 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":103,"threads_started":1}
I20260807 15:10:37.454708 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.450287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:38351 (local address 127.0.0.1:41540)
0807 15:10:37.450418 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.450451 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:37.450516 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:37.451050 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:37.451051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:37.451053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:37.451135 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.451137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.452514 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.452515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.453475 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.453477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.454323 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.454324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.454387 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.454391 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:37.454431 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:10:37.454474 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260807 15:10:37.454872 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.450677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:40469 (local address 127.0.0.1:46354)
0807 15:10:37.450788 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.450796 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:37.450831 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:37.451308 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:37.451309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:37.451311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:37.451344 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.451345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.452577 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.452578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.453509 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.453511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.454471 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.454472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.454521 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.454525 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:37.454754 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 15:10:37.454792 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260807 15:10:37.454981 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.450623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.18.129.1:37711)
0807 15:10:37.450831 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.450836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.450868 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:10:37.450909 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.450911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.450926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.451023 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:37.451238 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.451993 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.453280 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.454017 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.454090 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.454184 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.454824 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.454893 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260807 15:10:37.455103 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.450782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46354 (local address 127.18.129.3:40469)
0807 15:10:37.450988 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.450991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.451000 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:10:37.451157 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.451159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.451173 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.451243 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:37.451422 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.452492 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.453599 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.454292 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.454356 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.454441 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.454965 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.455006 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
I20260807 15:10:37.455598 13627 tablet_service.cc:1467] Processing CreateTablet for tablet 0169a63563fa49309aa53f4a0e73974e (DEFAULT_TABLE table=test-table [id=77d296c8cea34a1f9289d8885b0cee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:37.455650 13416 tablet_service.cc:1467] Processing CreateTablet for tablet 0169a63563fa49309aa53f4a0e73974e (DEFAULT_TABLE table=test-table [id=77d296c8cea34a1f9289d8885b0cee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:37.455724 13830 tablet_service.cc:1467] Processing CreateTablet for tablet 0169a63563fa49309aa53f4a0e73974e (DEFAULT_TABLE table=test-table [id=77d296c8cea34a1f9289d8885b0cee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:37.455997 13416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0169a63563fa49309aa53f4a0e73974e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:37.455997 13627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0169a63563fa49309aa53f4a0e73974e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:37.456089 13830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0169a63563fa49309aa53f4a0e73974e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:37.458730 13939 tablet_bootstrap.cc:492] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Bootstrap starting.
I20260807 15:10:37.459476 13941 tablet_bootstrap.cc:492] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2: Bootstrap starting.
I20260807 15:10:37.459514 13939 tablet_bootstrap.cc:654] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:37.459949 13939 log.cc:824] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:37.460206 13941 tablet_bootstrap.cc:654] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:37.460549 13941 log.cc:824] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:37.460741 13939 tablet_bootstrap.cc:492] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: No bootstrap required, opened a new log
I20260807 15:10:37.460816 13939 ts_tablet_manager.cc:1397] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:10:37.460947 13940 tablet_bootstrap.cc:492] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Bootstrap starting.
I20260807 15:10:37.461285 13941 tablet_bootstrap.cc:492] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2: No bootstrap required, opened a new log
I20260807 15:10:37.461357 13941 ts_tablet_manager.cc:1397] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:37.461623 13940 tablet_bootstrap.cc:654] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:37.461973 13940 log.cc:824] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:37.462790 13940 tablet_bootstrap.cc:492] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: No bootstrap required, opened a new log
I20260807 15:10:37.462888 13940 ts_tablet_manager.cc:1397] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:37.463425 13939 raft_consensus.cc:348] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.463582 13939 raft_consensus.cc:374] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:37.463626 13939 raft_consensus.cc:729] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260e31f0493b4faf8348b1b5ef2dea87, State: Initialized, Role: FOLLOWER
I20260807 15:10:37.463770 13939 consensus_queue.cc:260] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.463752 13941 raft_consensus.cc:348] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.463901 13941 raft_consensus.cc:374] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:37.463939 13941 raft_consensus.cc:729] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54e6415196984c34bd92932cc57639f2, State: Initialized, Role: FOLLOWER
I20260807 15:10:37.464053 13939 ts_tablet_manager.cc:1428] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 15:10:37.464082 13941 consensus_queue.cc:260] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.464188 13916 heartbeater.cc:503] Master 127.18.129.62:34065 was elected leader, sending a full tablet report...
I20260807 15:10:37.464341 13941 ts_tablet_manager.cc:1428] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:37.464489 13525 heartbeater.cc:503] Master 127.18.129.62:34065 was elected leader, sending a full tablet report...
I20260807 15:10:37.465355 13940 raft_consensus.cc:348] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.465512 13940 raft_consensus.cc:374] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:37.465555 13940 raft_consensus.cc:729] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1db3f6d10a469f81b87c77580f5ec0, State: Initialized, Role: FOLLOWER
I20260807 15:10:37.465665 13940 consensus_queue.cc:260] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.465899 13940 ts_tablet_manager.cc:1428] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:37.465991 13728 heartbeater.cc:503] Master 127.18.129.62:34065 was elected leader, sending a full tablet report...
W20260807 15:10:37.544876 13729 tablet.cc:2369] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:10:37.659821 13526 tablet.cc:2369] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:10:37.676859 13918 tablet.cc:2369] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:37.717836 13947 raft_consensus.cc:482] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:37.717985 13947 raft_consensus.cc:504] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.718321 13947 leader_election.cc:288] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1db3f6d10a469f81b87c77580f5ec0 (127.18.129.2:38351), 260e31f0493b4faf8348b1b5ef2dea87 (127.18.129.3:40469)
I20260807 15:10:37.721572 13951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.718675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:40469 (local address 127.18.129.1:58403)
0807 15:10:37.718776 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.718785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:37.718836 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:37.719021 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:37.719021 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:37.719023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:37.719125 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.719128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.719762 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.719763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.720502 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.720505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.721236 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.721237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.721299 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.721303 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:37.721345 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:10:37.721475 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":44,"threads_started":1}
I20260807 15:10:37.721560 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.718783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:58403 (local address 127.18.129.3:40469)
0807 15:10:37.718829 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.718831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.718841 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:10:37.718943 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.718944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.718953 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.719010 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:37.719195 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.719677 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.720598 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.721158 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.721231 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.721239 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.721372 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.721449 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:10:37.721711 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.718512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:38351 (local address 127.18.129.1:52751)
0807 15:10:37.718656 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.718670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:37.718753 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:37.718937 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:37.718939 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:37.718941 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:37.719030 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.719032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.719702 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.719703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.720496 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:37.720498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.721310 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.721311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:37.721377 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.721381 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:37.721586 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 15:10:37.721630 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260807 15:10:37.721733 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.718642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:52751 (local address 127.18.129.2:38351)
0807 15:10:37.718732 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.718734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.718747 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:10:37.718838 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.718839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.718867 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.718917 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:37.719097 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.719611 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.720603 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.721160 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.721231 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.721239 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.721622 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.721670 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:10:37.721918 13855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0169a63563fa49309aa53f4a0e73974e" candidate_uuid: "54e6415196984c34bd92932cc57639f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e31f0493b4faf8348b1b5ef2dea87" is_pre_election: true
I20260807 15:10:37.722081 13855 raft_consensus.cc:2454] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54e6415196984c34bd92932cc57639f2 in term 0.
I20260807 15:10:37.722077 13657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0169a63563fa49309aa53f4a0e73974e" candidate_uuid: "54e6415196984c34bd92932cc57639f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" is_pre_election: true
I20260807 15:10:37.722245 13657 raft_consensus.cc:2454] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54e6415196984c34bd92932cc57639f2 in term 0.
I20260807 15:10:37.722332 13350 leader_election.cc:302] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 260e31f0493b4faf8348b1b5ef2dea87, 54e6415196984c34bd92932cc57639f2; no voters:
I20260807 15:10:37.722457 13947 raft_consensus.cc:2781] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:37.722515 13947 raft_consensus.cc:482] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:37.722532 13947 raft_consensus.cc:3037] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:37.723326 13947 raft_consensus.cc:504] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.723443 13947 leader_election.cc:288] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [CANDIDATE]: Term 1 election: Requested vote from peers cc1db3f6d10a469f81b87c77580f5ec0 (127.18.129.2:38351), 260e31f0493b4faf8348b1b5ef2dea87 (127.18.129.3:40469)
I20260807 15:10:37.723628 13855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0169a63563fa49309aa53f4a0e73974e" candidate_uuid: "54e6415196984c34bd92932cc57639f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260e31f0493b4faf8348b1b5ef2dea87"
I20260807 15:10:37.723639 13657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0169a63563fa49309aa53f4a0e73974e" candidate_uuid: "54e6415196984c34bd92932cc57639f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1db3f6d10a469f81b87c77580f5ec0"
I20260807 15:10:37.723698 13855 raft_consensus.cc:3037] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:37.723718 13657 raft_consensus.cc:3037] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:37.724386 13855 raft_consensus.cc:2454] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54e6415196984c34bd92932cc57639f2 in term 1.
I20260807 15:10:37.724524 13657 raft_consensus.cc:2454] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54e6415196984c34bd92932cc57639f2 in term 1.
I20260807 15:10:37.724567 13350 leader_election.cc:302] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 260e31f0493b4faf8348b1b5ef2dea87, 54e6415196984c34bd92932cc57639f2; no voters:
I20260807 15:10:37.724675 13947 raft_consensus.cc:2781] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:37.724828 13947 raft_consensus.cc:686] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [term 1 LEADER]: Becoming Leader. State: Replica: 54e6415196984c34bd92932cc57639f2, State: Running, Role: LEADER
I20260807 15:10:37.724936 13947 consensus_queue.cc:237] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } }
I20260807 15:10:37.725797 13234 catalog_manager.cc:5495] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54e6415196984c34bd92932cc57639f2 (127.18.129.1). New cstate: current_term: 1 leader_uuid: "54e6415196984c34bd92932cc57639f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54e6415196984c34bd92932cc57639f2" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 37711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 } health_report { overall_health: UNKNOWN } } }
I20260807 15:10:37.739675 13935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.737337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.18.129.1:37711)
0807 15:10:37.737390 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:37.737392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:37.737401 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:10:37.737459 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:37.737460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:37.737480 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:37.737539 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:37.737689 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.738166 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.738939 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:37.739140 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:37.739206 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:37.739358 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:37.739464 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:37.739537 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:37.739575 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 15:10:37.739597 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:10:37.741631 13855 raft_consensus.cc:1261] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 1 FOLLOWER]: Refusing update from remote peer 54e6415196984c34bd92932cc57639f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:37.741631 13657 raft_consensus.cc:1261] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 1 FOLLOWER]: Refusing update from remote peer 54e6415196984c34bd92932cc57639f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:37.742038 13953 consensus_queue.cc:1035] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "260e31f0493b4faf8348b1b5ef2dea87" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 40469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:37.742192 13947 consensus_queue.cc:1035] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1db3f6d10a469f81b87c77580f5ec0" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:37.744860 13959 mvcc.cc:205] Tried to move back new op lower bound from 7315928832986734592 to 7315928832922030080. Current Snapshot: MvccSnapshot[applied={T|T < 7315928832986734592}]
I20260807 15:10:37.744889 13960 mvcc.cc:205] Tried to move back new op lower bound from 7315928832986734592 to 7315928832922030080. Current Snapshot: MvccSnapshot[applied={T|T < 7315928832986734592}]
I20260807 15:10:37.745486 13962 mvcc.cc:205] Tried to move back new op lower bound from 7315928832986734592 to 7315928832922030080. Current Snapshot: MvccSnapshot[applied={T|T < 7315928832986734592}]
I20260807 15:10:37.747913 13234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:37.748003 13234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:37.748858 13234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97beb77c269d4eb5a65bc9d1408d5f7c: Sending DeleteTablet for 3 replicas of tablet 0169a63563fa49309aa53f4a0e73974e
I20260807 15:10:37.749189 13627 tablet_service.cc:1514] Processing DeleteTablet for tablet 0169a63563fa49309aa53f4a0e73974e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:37 UTC) from {username='slave'} at 127.0.0.1:41540
I20260807 15:10:37.749272 13416 tablet_service.cc:1514] Processing DeleteTablet for tablet 0169a63563fa49309aa53f4a0e73974e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:37 UTC) from {username='slave'} at 127.0.0.1:52048
I20260807 15:10:37.749483 13976 tablet_replica.cc:330] T 0169a63563fa49309aa53f4a0e73974e P 54e6415196984c34bd92932cc57639f2: stopping tablet replica
I20260807 15:10:37.749507 13830 tablet_service.cc:1514] Processing DeleteTablet for tablet 0169a63563fa49309aa53f4a0e73974e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:37 UTC) from {username='slave'} at 127.0.0.1:46354
I20260807 15:10:37.749733 13975 tablet_replica.cc:330] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: stopping tablet replica
I20260807 15:10:37.749843 13975 raft_consensus.cc:2229] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:37.749918 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 13325
I20260807 15:10:37.749950 13975 raft_consensus.cc:2258] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:37.753361 13975 ts_tablet_manager.cc:1905] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:37.755041 13978 tablet_replica.cc:330] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: stopping tablet replica
I20260807 15:10:37.755163 13975 ts_tablet_manager.cc:1918] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:37.755185 13978 raft_consensus.cc:2229] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:37.755230 13975 log.cc:1196] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/0169a63563fa49309aa53f4a0e73974e
I20260807 15:10:37.755300 13978 raft_consensus.cc:2258] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:37.755476 13975 ts_tablet_manager.cc:1939] T 0169a63563fa49309aa53f4a0e73974e P cc1db3f6d10a469f81b87c77580f5ec0: Deleting consensus metadata
I20260807 15:10:37.755862 13215 catalog_manager.cc:4841] TS cc1db3f6d10a469f81b87c77580f5ec0 (127.18.129.2:38351): tablet 0169a63563fa49309aa53f4a0e73974e (table test-table [id=77d296c8cea34a1f9289d8885b0cee85]) successfully deleted
I20260807 15:10:37.756736 13978 ts_tablet_manager.cc:1905] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:37.758299 13978 ts_tablet_manager.cc:1918] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:37.758371 13978 log.cc:1196] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/0169a63563fa49309aa53f4a0e73974e
I20260807 15:10:37.758574 13978 ts_tablet_manager.cc:1939] T 0169a63563fa49309aa53f4a0e73974e P 260e31f0493b4faf8348b1b5ef2dea87: Deleting consensus metadata
I20260807 15:10:37.759037 13215 catalog_manager.cc:4841] TS 260e31f0493b4faf8348b1b5ef2dea87 (127.18.129.3:40469): tablet 0169a63563fa49309aa53f4a0e73974e (table test-table [id=77d296c8cea34a1f9289d8885b0cee85]) successfully deleted
W20260807 15:10:37.760905 13215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:37711 (error 108)
I20260807 15:10:37.761111 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 13528
I20260807 15:10:37.764819 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:37.761341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:37711 (local address 127.0.0.1:52080)
0807 15:10:37.764702 (+ 3361us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:37.764776 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:37711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3332}
W20260807 15:10:37.766093 13215 catalog_manager.cc:4584] TS 54e6415196984c34bd92932cc57639f2 (127.18.129.1:37711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0169a63563fa49309aa53f4a0e73974e: Network error: Client connection negotiation failed: client connection to 127.18.129.1:37711: connect: Connection refused (error 111)
I20260807 15:10:37.775252 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 13731
I20260807 15:10:37.786455 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 13185
2026-08-07T15:10:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (948 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:37 dist-test-slave-5bn2 krb5kdc[13984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:37 dist-test-slave-5bn2 krb5kdc[13984](info): set up 2 sockets
Aug 07 15:10:37 dist-test-slave-5bn2 krb5kdc[13984](info): commencing operation
krb5kdc: starting...
W20260807 15:10:38.910063 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:38 dist-test-slave-5bn2 krb5kdc[13984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:10:39.031527 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:35541
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:35541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:39.116441 14010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:39.116643 14010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:39.116674 14010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:39.117805 14010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:39.117846 14010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:39.117864 14010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:39.117880 14010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:39.117894 14010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:39.119396 14010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:35541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:35541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:39.119637 14010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:39.119835 14010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:39.124732 14020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:39.124745 14024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:39.124886 14010 server_base.cc:1034] running on GCE node
W20260807 15:10:39.124836 14021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:39.125247 14010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:39.125494 14010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:39.126645 14010 hybrid_clock.cc:648] HybridClock initialized: now 1786115439126635 us; error 35 us; skew 500 ppm
Aug 07 15:10:39 dist-test-slave-5bn2 krb5kdc[13984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:39.129079 14010 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:10:39.129514 14010 webserver.cc:460] Webserver started at http://127.18.129.62:42845/ using document root <none> and password file <none>
I20260807 15:10:39.129693 14010 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:39.129729 14010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:39.129819 14010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:39.130822 14010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "6e723c35ba184b34b3da559c11c549f2"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "35ab2de2869b9d7fbd4e0b0b06923b6f"
server_key_iv: "60c37e60a95669ccf45ddbbfc1e94dea"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.131177 14010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "6e723c35ba184b34b3da559c11c549f2"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "35ab2de2869b9d7fbd4e0b0b06923b6f"
server_key_iv: "60c37e60a95669ccf45ddbbfc1e94dea"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.132383 14010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:39.133059 14034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.133221 14010 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:39.133307 14010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "6e723c35ba184b34b3da559c11c549f2"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "35ab2de2869b9d7fbd4e0b0b06923b6f"
server_key_iv: "60c37e60a95669ccf45ddbbfc1e94dea"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.133368 14010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:39.149129 14010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:39.151142 14010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:39.151286 14010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:39.155290 14010 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:35541
I20260807 15:10:39.155329 14108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:35541 every 8 connection(s)
I20260807 15:10:39.155678 14010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:39.156481 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 14010
I20260807 15:10:39.156622 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:39.156665 14109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:39.156837 18948 external_mini_cluster.cc:1428] Setting key 1f8107c8acb1b755976421212cb81145
I20260807 15:10:39.159746 14109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2: Bootstrap starting.
I20260807 15:10:39.160434 14109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:39.160776 14109 log.cc:824] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:39.161548 14109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2: No bootstrap required, opened a new log
Aug 07 15:10:39 dist-test-slave-5bn2 krb5kdc[13984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:39.163805 14109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e723c35ba184b34b3da559c11c549f2" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35541 } }
I20260807 15:10:39.163990 14109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:39.164048 14109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e723c35ba184b34b3da559c11c549f2, State: Initialized, Role: FOLLOWER
I20260807 15:10:39.164186 14109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e723c35ba184b34b3da559c11c549f2" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35541 } }
I20260807 15:10:39.164263 14109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:39.164294 14109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:39.164340 14109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:39.165016 14109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e723c35ba184b34b3da559c11c549f2" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35541 } }
I20260807 15:10:39.165129 14109 leader_election.cc:302] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e723c35ba184b34b3da559c11c549f2; no voters:
I20260807 15:10:39.165293 14109 leader_election.cc:288] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:39.165345 14114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:39.165490 14114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [term 1 LEADER]: Becoming Leader. State: Replica: 6e723c35ba184b34b3da559c11c549f2, State: Running, Role: LEADER
I20260807 15:10:39.165578 14109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:39.165642 14114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e723c35ba184b34b3da559c11c549f2" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35541 } }
I20260807 15:10:39.165946 14114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e723c35ba184b34b3da559c11c549f2. Latest consensus state: current_term: 1 leader_uuid: "6e723c35ba184b34b3da559c11c549f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e723c35ba184b34b3da559c11c549f2" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35541 } } }
I20260807 15:10:39.166039 14114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:39.165961 14118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e723c35ba184b34b3da559c11c549f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e723c35ba184b34b3da559c11c549f2" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35541 } } }
I20260807 15:10:39.166210 14118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:39.166244 14127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:39.166740 14127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:39.168285 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.157371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.18.129.62:35541)
0807 15:10:39.157558 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.157565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.157595 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:10:39.158078 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.158085 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.158132 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.158242 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:39.159063 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.159982 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.161037 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.161635 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.161745 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.164205 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:39.164209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:39.164209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:39.164222 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:39.166115 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.166547 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.166550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.166551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.166669 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.166979 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.166979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.166980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.167285 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:39.167854 (+ 569us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.167941 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.168092 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260807 15:10:39.168433 14127 catalog_manager.cc:1347] Generated new cluster ID: 1df54c3ed15742c3ad40c1fd3396031a
I20260807 15:10:39.168476 14127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:39.184188 14127 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:39.184805 14127 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:39.191790 14127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2: Generated new TSK 0
I20260807 15:10:39.191998 14127 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:10:39.225447 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35541
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:39.315038 14145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:39.315241 14145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:39.315270 14145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:39.316401 14145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:39.316442 14145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:39.316475 14145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:10:39.317899 14145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:39.318126 14145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:39.318326 14145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:39.323333 14157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:39.323340 14154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:39.323549 14145 server_base.cc:1034] running on GCE node
W20260807 15:10:39.323369 14152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:39.323813 14145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:39.324023 14145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:39.325152 14145 hybrid_clock.cc:648] HybridClock initialized: now 1786115439325140 us; error 27 us; skew 500 ppm
Aug 07 15:10:39 dist-test-slave-5bn2 krb5kdc[13984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:39.327549 14145 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:10:39.327955 14145 webserver.cc:460] Webserver started at http://127.18.129.1:38593/ using document root <none> and password file <none>
I20260807 15:10:39.328159 14145 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:39.328209 14145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:39.328339 14145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:39.329396 14145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "7594b5c621f54cfba903f99bdf96a405"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "c270a605c5d7600e71c03111f58a4f75"
server_key_iv: "9cd2a406b3c1779c59b308d52ae2dfe4"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.329746 14145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "7594b5c621f54cfba903f99bdf96a405"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "c270a605c5d7600e71c03111f58a4f75"
server_key_iv: "9cd2a406b3c1779c59b308d52ae2dfe4"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.331046 14145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:39.331811 14170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.332001 14145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:39.332101 14145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "7594b5c621f54cfba903f99bdf96a405"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "c270a605c5d7600e71c03111f58a4f75"
server_key_iv: "9cd2a406b3c1779c59b308d52ae2dfe4"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.332166 14145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:39.355165 14145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:39.357111 14145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:39.357273 14145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:39.357539 14145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:39.357887 14145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:39.357926 14145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.357960 14145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:39.357987 14145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.364284 14145 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:40667
I20260807 15:10:39.364327 14347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:40667 every 8 connection(s)
I20260807 15:10:39.364670 14145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:10:39 dist-test-slave-5bn2 krb5kdc[13984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:39.371707 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 14145
I20260807 15:10:39.371824 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:39.372018 18948 external_mini_cluster.cc:1428] Setting key e85a8c2feffd4a245bea1b3bdfa0655f
I20260807 15:10:39.374150 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.365394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:33269 (local address 127.18.129.62:35541)
0807 15:10:39.365440 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.365442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.365654 (+ 212us) server_negotiation.cc:407] Connection header received
0807 15:10:39.366196 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.366197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.366241 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.366323 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:39.367442 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.367960 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.368978 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.369195 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.369260 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.371441 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:39.371444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:39.371444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:39.371451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:39.372897 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.373270 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.373272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.373272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.373325 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.373537 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.373537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.373538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.373648 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:39.373702 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.373965 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.374059 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:10:39.374315 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.365372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35541 (local address 127.18.129.1:33269)
0807 15:10:39.365539 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.365582 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:39.366091 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:39.366422 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:39.366426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:39.367099 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:39.367302 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.367308 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.368051 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.368052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.368848 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.368850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.369267 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.369267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.369366 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.369852 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:39.369860 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:39.371339 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:39.372979 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:39.372986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:39.372988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:39.373204 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:39.373378 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:39.373378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:39.373379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:39.373482 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:39.373701 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:39.373708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:39.373845 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 15:10:39.373936 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:10:39.374096 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260807 15:10:39.374756 14348 heartbeater.cc:347] Connected to a master server at 127.18.129.62:35541
I20260807 15:10:39.374873 14348 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:39.375059 14348 heartbeater.cc:511] Master 127.18.129.62:35541 requested a full tablet report, sending...
I20260807 15:10:39.375614 14057 ts_manager.cc:194] Registered new tserver with Master: 7594b5c621f54cfba903f99bdf96a405 (127.18.129.1:40667)
I20260807 15:10:39.376313 14057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:33269
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:10:39.397710 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35541
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:39.480892 14357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:39.481114 14357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:39.481145 14357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:39.482261 14357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:39.482303 14357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:39.482335 14357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:10:39.483780 14357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:39.484010 14357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:39.484222 14357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:39.489262 14370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:39.489218 14368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:39.489209 14367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:39.489249 14357 server_base.cc:1034] running on GCE node
I20260807 15:10:39.489625 14357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:39.489866 14357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:39.491007 14357 hybrid_clock.cc:648] HybridClock initialized: now 1786115439490988 us; error 35 us; skew 500 ppm
Aug 07 15:10:39 dist-test-slave-5bn2 krb5kdc[13984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:39.496361 14357 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:10:39.496830 14357 webserver.cc:460] Webserver started at http://127.18.129.2:35895/ using document root <none> and password file <none>
I20260807 15:10:39.497031 14357 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:39.497079 14357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:39.497200 14357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:39.498214 14357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "aaeb008fa43845990dcc80dfe3818830"
server_key_iv: "55c187e60d4618439b336d675b4640f0"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.498551 14357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "aaeb008fa43845990dcc80dfe3818830"
server_key_iv: "55c187e60d4618439b336d675b4640f0"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.499828 14357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:39.500483 14380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.500631 14357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:39.500715 14357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "aaeb008fa43845990dcc80dfe3818830"
server_key_iv: "55c187e60d4618439b336d675b4640f0"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.500772 14357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:39.509938 14357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:39.511628 14357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:39.511757 14357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:39.511963 14357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:39.512251 14357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:39.512284 14357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.512311 14357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:39.512328 14357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.517808 14357 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:37909
I20260807 15:10:39.517870 14571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:37909 every 8 connection(s)
I20260807 15:10:39.518148 14357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:39.522964 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 14357
I20260807 15:10:39.523082 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:39.523267 18948 external_mini_cluster.cc:1428] Setting key 80c12aa58e126fb327e6aaf5c9aba21a
Aug 07 15:10:39 dist-test-slave-5bn2 krb5kdc[13984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:39.527784 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.518788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:45085 (local address 127.18.129.62:35541)
0807 15:10:39.518840 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.518842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.519047 (+ 205us) server_negotiation.cc:407] Connection header received
0807 15:10:39.519518 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.519519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.519530 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.519587 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:39.520512 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.521019 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.521968 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.522174 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.522227 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.525109 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:39.525110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:39.525111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:39.525119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:39.526451 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.526865 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.526866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.526867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.526930 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.527151 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.527152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.527152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.527268 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:39.527325 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.527598 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.527693 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:10:39.527952 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.518753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35541 (local address 127.18.129.2:45085)
0807 15:10:39.518921 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.518963 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:39.519435 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:39.519667 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:39.519671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:39.520228 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:39.520419 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.520425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.521086 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.521086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.521892 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.521894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.522234 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.522235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.522331 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.523513 (+ 1182us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:39.523524 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:39.525018 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:39.526536 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:39.526544 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:39.526546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:39.526772 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:39.526985 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:39.526986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:39.526987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:39.527095 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:39.527323 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:39.527329 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:39.527471 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 15:10:39.527567 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:10:39.527734 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260807 15:10:39.528409 14572 heartbeater.cc:347] Connected to a master server at 127.18.129.62:35541
I20260807 15:10:39.528537 14572 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:39.528734 14572 heartbeater.cc:511] Master 127.18.129.62:35541 requested a full tablet report, sending...
I20260807 15:10:39.529140 14057 ts_manager.cc:194] Registered new tserver with Master: 8a1a7494fb7a4c9f94ba2a219c1d84af (127.18.129.2:37909)
I20260807 15:10:39.529532 14057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:45085
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:10:39.549593 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35541
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:39.636951 14579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:39.637239 14579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:39.637277 14579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:39.638999 14579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:39.639062 14579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:39.639104 14579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:10:39.641271 14579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:35541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:39.641546 14579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:39.641819 14579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:39.650345 14589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:39.650345 14585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:39.650658 14586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:39.651011 14579 server_base.cc:1034] running on GCE node
I20260807 15:10:39.651211 14579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:39.651432 14579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:39.652594 14579 hybrid_clock.cc:648] HybridClock initialized: now 1786115439652575 us; error 31 us; skew 500 ppm
Aug 07 15:10:39 dist-test-slave-5bn2 krb5kdc[13984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:39.655992 14579 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:10:39.658972 14579 webserver.cc:460] Webserver started at http://127.18.129.3:41829/ using document root <none> and password file <none>
I20260807 15:10:39.659246 14579 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:39.659322 14579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:39.659476 14579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:39.660899 14579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "bc94d0234c61410bb8b814da31ebb3cd"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "b32ecc4a0a7a22cef930679bb42d5e19"
server_key_iv: "083542af6f83770ffa68f3d45c081a0d"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.661332 14579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "bc94d0234c61410bb8b814da31ebb3cd"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "b32ecc4a0a7a22cef930679bb42d5e19"
server_key_iv: "083542af6f83770ffa68f3d45c081a0d"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.662936 14579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:10:39.663749 14600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.663928 14579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:39.664063 14579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "bc94d0234c61410bb8b814da31ebb3cd"
format_stamp: "Formatted at 2026-08-07 15:10:39 on dist-test-slave-5bn2"
server_key: "b32ecc4a0a7a22cef930679bb42d5e19"
server_key_iv: "083542af6f83770ffa68f3d45c081a0d"
server_key_version: "encryptionkey@0"
I20260807 15:10:39.664141 14579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:39.700071 14579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:39.702183 14579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:39.702316 14579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:39.702571 14579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:39.702921 14579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:39.702963 14579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.702996 14579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:39.703019 14579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:39.708990 14579 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:35747
I20260807 15:10:39.709074 14782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:35747 every 8 connection(s)
I20260807 15:10:39.709365 14579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:10:39 dist-test-slave-5bn2 krb5kdc[13984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:39.715713 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 14579
I20260807 15:10:39.715821 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:39.715971 18948 external_mini_cluster.cc:1428] Setting key 9904e660205008e4d31a4db19e077433
I20260807 15:10:39.718421 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.710070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:35657 (local address 127.18.129.62:35541)
0807 15:10:39.710132 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.710134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.710346 (+ 212us) server_negotiation.cc:407] Connection header received
0807 15:10:39.710908 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.710910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.710923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.710968 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:39.711915 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.712411 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.713414 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.713611 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.713671 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.715778 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:39.715780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:39.715780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:39.715788 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:39.717270 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.717632 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.717633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.717633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.717678 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.717845 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.717845 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.717845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.717937 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:39.717981 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.718265 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.718343 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:10:39.718497 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.710038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35541 (local address 127.18.129.3:35657)
0807 15:10:39.710225 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.710262 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:39.710788 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:39.711047 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:39.711050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:39.711623 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:39.711819 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.711823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.712508 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.712509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.713328 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.713330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.713683 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.713684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.713803 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.714277 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:39.714285 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:39.715685 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:39.717336 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:39.717343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:39.717345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:39.717572 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:39.717718 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:39.717718 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:39.717719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:39.717807 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:39.717980 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:39.717982 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:39.718118 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 15:10:39.718201 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:10:39.718323 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":106,"threads_started":1}
I20260807 15:10:39.718986 14785 heartbeater.cc:347] Connected to a master server at 127.18.129.62:35541
I20260807 15:10:39.719086 14785 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:39.719272 14785 heartbeater.cc:511] Master 127.18.129.62:35541 requested a full tablet report, sending...
I20260807 15:10:39.719651 14057 ts_manager.cc:194] Registered new tserver with Master: bc94d0234c61410bb8b814da31ebb3cd (127.18.129.3:35747)
I20260807 15:10:39.720014 14057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:35657
I20260807 15:10:39.727648 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:10:39.734643 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.728926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47824 (local address 127.18.129.62:35541)
0807 15:10:39.728987 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.728989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.729073 (+ 84us) server_negotiation.cc:407] Connection header received
0807 15:10:39.729138 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.729139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.729149 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.729195 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:39.729757 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.730244 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.730993 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.731185 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.731240 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.732447 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:39.732448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:39.732448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:39.732456 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:39.733451 (+ 995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.733831 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.733832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.733832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.733885 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:39.734079 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:39.734080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:39.734080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:39.734258 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:39.734354 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.734492 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.734552 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:10:39.736169 14057 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:39.737841 14057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:39.744036 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.740035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:37909 (local address 127.0.0.1:57948)
0807 15:10:39.740129 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.740167 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:39.740235 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:39.740807 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:39.740810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:39.740813 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:39.740889 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.740892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.741768 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.741769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.742992 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.742995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.743741 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.743742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.743822 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.743836 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:10:39.743901 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:10:39.743952 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":56,"threads_started":1}
I20260807 15:10:39.744045 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.740185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.18.129.2:37909)
0807 15:10:39.740379 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.740384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.740394 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:10:39.740430 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.740432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.740450 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.740562 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:39.740978 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.741671 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.743079 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.743656 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.743723 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.743787 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.743923 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.743966 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":124,"threads_started":1}
I20260807 15:10:39.744369 14813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.740383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:40667 (local address 127.0.0.1:45192)
0807 15:10:39.740612 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.740621 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:39.740661 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:39.740965 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:39.740966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:39.740968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:39.740996 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.740997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.742241 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.742242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.743191 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.743194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.744140 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.744141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.744205 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.744210 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:10:39.744240 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 15:10:39.744280 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":59,"threads_started":1}
I20260807 15:10:39.744520 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.740329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.18.129.3:35747)
0807 15:10:39.740518 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.740522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.740639 (+ 117us) server_negotiation.cc:407] Connection header received
0807 15:10:39.740687 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.740690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.740705 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.740776 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:39.740968 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.742082 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.743311 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.744022 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.744093 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.744152 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.744409 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.744450 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260807 15:10:39.744371 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.740217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:35747 (local address 127.0.0.1:46834)
0807 15:10:39.740514 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.740524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:39.740579 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:39.740840 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:39.740841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:39.740844 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:39.740900 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.740902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.742198 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.742199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.743227 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.743230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.744180 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.744181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.744233 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.744236 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:39.744277 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 15:10:39.744306 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":85,"threads_started":1}
I20260807 15:10:39.744706 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.740613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45192 (local address 127.18.129.1:40667)
0807 15:10:39.740807 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.740812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.740821 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:10:39.740856 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.740857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.740871 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.741063 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:39.741131 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.742162 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.743275 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.744030 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.744487 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.744557 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.744583 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.744630 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260807 15:10:39.744869 14468 tablet_service.cc:1467] Processing CreateTablet for tablet 05823591133645338744c879d9c4c13d (DEFAULT_TABLE table=test-table [id=7ba20a741181427a94befec216472567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:39.745276 14468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05823591133645338744c879d9c4c13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:39.747553 14247 tablet_service.cc:1467] Processing CreateTablet for tablet 05823591133645338744c879d9c4c13d (DEFAULT_TABLE table=test-table [id=7ba20a741181427a94befec216472567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:39.747658 14674 tablet_service.cc:1467] Processing CreateTablet for tablet 05823591133645338744c879d9c4c13d (DEFAULT_TABLE table=test-table [id=7ba20a741181427a94befec216472567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:39.747888 14247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05823591133645338744c879d9c4c13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:39.747998 14674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05823591133645338744c879d9c4c13d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:39.748147 14818 tablet_bootstrap.cc:492] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Bootstrap starting.
I20260807 15:10:39.748909 14818 tablet_bootstrap.cc:654] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:39.749528 14818 log.cc:824] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:39.750442 14818 tablet_bootstrap.cc:492] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: No bootstrap required, opened a new log
I20260807 15:10:39.750517 14818 ts_tablet_manager.cc:1397] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:39.750722 14821 tablet_bootstrap.cc:492] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: Bootstrap starting.
I20260807 15:10:39.751228 14824 tablet_bootstrap.cc:492] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405: Bootstrap starting.
I20260807 15:10:39.751353 14821 tablet_bootstrap.cc:654] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:39.751895 14821 log.cc:824] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:39.751952 14824 tablet_bootstrap.cc:654] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:39.752287 14818 raft_consensus.cc:348] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.752442 14824 log.cc:824] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:39.752461 14818 raft_consensus.cc:374] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:39.752507 14818 raft_consensus.cc:729] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a7494fb7a4c9f94ba2a219c1d84af, State: Initialized, Role: FOLLOWER
I20260807 15:10:39.752620 14818 consensus_queue.cc:260] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.752732 14821 tablet_bootstrap.cc:492] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: No bootstrap required, opened a new log
I20260807 15:10:39.752792 14821 ts_tablet_manager.cc:1397] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:39.752926 14818 ts_tablet_manager.cc:1428] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:10:39.752970 14572 heartbeater.cc:503] Master 127.18.129.62:35541 was elected leader, sending a full tablet report...
I20260807 15:10:39.753124 14824 tablet_bootstrap.cc:492] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405: No bootstrap required, opened a new log
I20260807 15:10:39.753168 14824 ts_tablet_manager.cc:1397] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:39.754547 14821 raft_consensus.cc:348] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.754674 14821 raft_consensus.cc:374] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:39.754705 14821 raft_consensus.cc:729] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc94d0234c61410bb8b814da31ebb3cd, State: Initialized, Role: FOLLOWER
I20260807 15:10:39.754809 14821 consensus_queue.cc:260] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.754837 14824 raft_consensus.cc:348] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.755030 14824 raft_consensus.cc:374] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:39.755074 14824 raft_consensus.cc:729] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7594b5c621f54cfba903f99bdf96a405, State: Initialized, Role: FOLLOWER
I20260807 15:10:39.755110 14785 heartbeater.cc:503] Master 127.18.129.62:35541 was elected leader, sending a full tablet report...
I20260807 15:10:39.755182 14824 consensus_queue.cc:260] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.755105 14821 ts_tablet_manager.cc:1428] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:39.755455 14824 ts_tablet_manager.cc:1428] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:10:39.755496 14348 heartbeater.cc:503] Master 127.18.129.62:35541 was elected leader, sending a full tablet report...
W20260807 15:10:39.768652 14573 tablet.cc:2369] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:10:39.865409 14349 tablet.cc:2369] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:39.900130 14829 raft_consensus.cc:482] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:39.900245 14829 raft_consensus.cc:504] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.900616 14829 leader_election.cc:288] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc94d0234c61410bb8b814da31ebb3cd (127.18.129.3:35747), 7594b5c621f54cfba903f99bdf96a405 (127.18.129.1:40667)
I20260807 15:10:39.904373 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.900901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:54883 (local address 127.18.129.3:35747)
0807 15:10:39.900955 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.900957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.900968 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:10:39.901001 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.901002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.901013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.901070 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:39.901233 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.902003 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.903471 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.904057 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.904118 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.904125 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.904264 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.904305 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:10:39.904373 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.900951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:43373 (local address 127.18.129.1:40667)
0807 15:10:39.900994 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.900995 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.901145 (+ 150us) server_negotiation.cc:407] Connection header received
0807 15:10:39.901177 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.901178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.901186 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.901234 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:39.901338 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.902036 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.903455 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.904009 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.904101 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.904112 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.904249 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.904295 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:10:39.904379 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.900778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:35747 (local address 127.18.129.2:54883)
0807 15:10:39.900831 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.900843 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:39.900904 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:39.901088 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:39.901089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:39.901091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:39.901168 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.901170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.902081 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.902082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.903370 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.903373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.904131 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.904132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.904203 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.904205 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:10:39.904250 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:10:39.904280 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":35}
I20260807 15:10:39.904386 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.900894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:40667 (local address 127.18.129.2:43373)
0807 15:10:39.901072 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.901082 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:39.901137 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:39.901236 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:39.901237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:39.901238 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:39.901288 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.901289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.902118 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.902118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.903374 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:39.903376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.904086 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.904087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:39.904180 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.904185 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:10:39.904241 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 15:10:39.904280 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":5,"thread_start_us":119,"threads_started":1}
I20260807 15:10:39.904811 14711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05823591133645338744c879d9c4c13d" candidate_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc94d0234c61410bb8b814da31ebb3cd" is_pre_election: true
I20260807 15:10:39.904788 14278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05823591133645338744c879d9c4c13d" candidate_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7594b5c621f54cfba903f99bdf96a405" is_pre_election: true
I20260807 15:10:39.904994 14278 raft_consensus.cc:2454] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a1a7494fb7a4c9f94ba2a219c1d84af in term 0.
I20260807 15:10:39.904994 14711 raft_consensus.cc:2454] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a1a7494fb7a4c9f94ba2a219c1d84af in term 0.
I20260807 15:10:39.905241 14384 leader_election.cc:302] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a1a7494fb7a4c9f94ba2a219c1d84af, bc94d0234c61410bb8b814da31ebb3cd; no voters:
I20260807 15:10:39.905368 14829 raft_consensus.cc:2781] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:39.905429 14829 raft_consensus.cc:482] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:39.905455 14829 raft_consensus.cc:3037] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:39.906152 14829 raft_consensus.cc:504] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.906266 14829 leader_election.cc:288] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [CANDIDATE]: Term 1 election: Requested vote from peers bc94d0234c61410bb8b814da31ebb3cd (127.18.129.3:35747), 7594b5c621f54cfba903f99bdf96a405 (127.18.129.1:40667)
I20260807 15:10:39.906446 14711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05823591133645338744c879d9c4c13d" candidate_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc94d0234c61410bb8b814da31ebb3cd"
I20260807 15:10:39.906474 14278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05823591133645338744c879d9c4c13d" candidate_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7594b5c621f54cfba903f99bdf96a405"
I20260807 15:10:39.906534 14711 raft_consensus.cc:3037] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:39.906543 14278 raft_consensus.cc:3037] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:39.907440 14711 raft_consensus.cc:2454] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a7494fb7a4c9f94ba2a219c1d84af in term 1.
I20260807 15:10:39.907440 14278 raft_consensus.cc:2454] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a7494fb7a4c9f94ba2a219c1d84af in term 1.
I20260807 15:10:39.907634 14384 leader_election.cc:302] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7594b5c621f54cfba903f99bdf96a405, 8a1a7494fb7a4c9f94ba2a219c1d84af; no voters:
I20260807 15:10:39.907723 14829 raft_consensus.cc:2781] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:39.907872 14829 raft_consensus.cc:686] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 1 LEADER]: Becoming Leader. State: Replica: 8a1a7494fb7a4c9f94ba2a219c1d84af, State: Running, Role: LEADER
I20260807 15:10:39.907953 14829 consensus_queue.cc:237] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } }
I20260807 15:10:39.908756 14056 catalog_manager.cc:5495] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a1a7494fb7a4c9f94ba2a219c1d84af (127.18.129.2). New cstate: current_term: 1 leader_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1a7494fb7a4c9f94ba2a219c1d84af" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 37909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 } health_report { overall_health: UNKNOWN } } }
I20260807 15:10:39.925470 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.923407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.18.129.2:37909)
0807 15:10:39.923460 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:39.923462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:39.923470 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:10:39.923500 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:39.923501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:39.923519 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:39.923576 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:39.923702 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.924156 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.924839 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:39.925032 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:39.925078 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:39.925176 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:39.925256 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:39.925319 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:39.925354 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 15:10:39.925362 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:10:39.927289 14711 raft_consensus.cc:1261] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 1 FOLLOWER]: Refusing update from remote peer 8a1a7494fb7a4c9f94ba2a219c1d84af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:39.927297 14278 raft_consensus.cc:1261] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405 [term 1 FOLLOWER]: Refusing update from remote peer 8a1a7494fb7a4c9f94ba2a219c1d84af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:39.927639 14835 consensus_queue.cc:1035] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc94d0234c61410bb8b814da31ebb3cd" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 35747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:39.927760 14829 consensus_queue.cc:1035] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7594b5c621f54cfba903f99bdf96a405" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 40667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:39.930437 14840 mvcc.cc:205] Tried to move back new op lower bound from 7315928841939451904 to 7315928841863491584. Current Snapshot: MvccSnapshot[applied={T|T < 7315928841939451904}]
I20260807 15:10:39.932020 14841 mvcc.cc:205] Tried to move back new op lower bound from 7315928841939451904 to 7315928841863491584. Current Snapshot: MvccSnapshot[applied={T|T < 7315928841939451904 or (T in {7315928841939451904})}]
I20260807 15:10:39.932369 14057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:39.932466 14057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:39.933240 14057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e723c35ba184b34b3da559c11c549f2: Sending DeleteTablet for 3 replicas of tablet 05823591133645338744c879d9c4c13d
I20260807 15:10:39.933539 14468 tablet_service.cc:1514] Processing DeleteTablet for tablet 05823591133645338744c879d9c4c13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:39 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:57948
I20260807 15:10:39.933589 14674 tablet_service.cc:1514] Processing DeleteTablet for tablet 05823591133645338744c879d9c4c13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:39 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:46834
I20260807 15:10:39.933681 14247 tablet_service.cc:1514] Processing DeleteTablet for tablet 05823591133645338744c879d9c4c13d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:39 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:45192
I20260807 15:10:39.933889 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 14145
I20260807 15:10:39.933916 14855 tablet_replica.cc:330] T 05823591133645338744c879d9c4c13d P 7594b5c621f54cfba903f99bdf96a405: stopping tablet replica
I20260807 15:10:39.934055 14852 tablet_replica.cc:330] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: stopping tablet replica
I20260807 15:10:39.934193 14852 raft_consensus.cc:2229] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:39.934360 14852 raft_consensus.cc:2258] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:39.934559 14854 tablet_replica.cc:330] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: stopping tablet replica
I20260807 15:10:39.934741 14854 raft_consensus.cc:2229] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:39.934931 14854 raft_consensus.cc:2258] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:39.936775 14852 ts_tablet_manager.cc:1905] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:39.938759 14852 ts_tablet_manager.cc:1918] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:39.938900 14852 log.cc:1196] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/05823591133645338744c879d9c4c13d
I20260807 15:10:39.939234 14852 ts_tablet_manager.cc:1939] T 05823591133645338744c879d9c4c13d P 8a1a7494fb7a4c9f94ba2a219c1d84af: Deleting consensus metadata
I20260807 15:10:39.940750 14038 catalog_manager.cc:4841] TS 8a1a7494fb7a4c9f94ba2a219c1d84af (127.18.129.2:37909): tablet 05823591133645338744c879d9c4c13d (table test-table [id=7ba20a741181427a94befec216472567]) successfully deleted
I20260807 15:10:39.940874 14854 ts_tablet_manager.cc:1905] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:39.942442 14854 ts_tablet_manager.cc:1918] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:39.942523 14854 log.cc:1196] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/05823591133645338744c879d9c4c13d
I20260807 15:10:39.942791 14854 ts_tablet_manager.cc:1939] T 05823591133645338744c879d9c4c13d P bc94d0234c61410bb8b814da31ebb3cd: Deleting consensus metadata
I20260807 15:10:39.943225 14038 catalog_manager.cc:4841] TS bc94d0234c61410bb8b814da31ebb3cd (127.18.129.3:35747): tablet 05823591133645338744c879d9c4c13d (table test-table [id=7ba20a741181427a94befec216472567]) successfully deleted
W20260807 15:10:39.948395 14038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:40667 (error 108)
I20260807 15:10:39.948613 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 14357
I20260807 15:10:39.948966 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:39.948850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:40667 (local address 127.0.0.1:45194)
0807 15:10:39.948903 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:39.948931 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:40667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 15:10:39.949071 14038 catalog_manager.cc:4584] TS 7594b5c621f54cfba903f99bdf96a405 (127.18.129.1:40667): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05823591133645338744c879d9c4c13d: Network error: Client connection negotiation failed: client connection to 127.18.129.1:40667: connect: Connection refused (error 111)
I20260807 15:10:39.956714 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 14579
I20260807 15:10:39.972359 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 14010
2026-08-07T15:10:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2187 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T15:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:39Z Disabled control of system clock
I20260807 15:10:40.002722 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43207
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:43207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:40.083067 14861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:40.083281 14861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:40.083302 14861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:40.084451 14861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:40.084496 14861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:40.084506 14861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:40.084517 14861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:40.084537 14861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:40.085855 14861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:43207
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:43207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:40.086149 14861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:40.086433 14861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:40.091357 14870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:40.091383 14861 server_base.cc:1034] running on GCE node
W20260807 15:10:40.091343 14871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:40.091406 14874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:40.091846 14861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:40.092172 14861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:40.093313 14861 hybrid_clock.cc:648] HybridClock initialized: now 1786115440093296 us; error 33 us; skew 500 ppm
I20260807 15:10:40.099795 14861 webserver.cc:460] Webserver started at http://127.18.129.62:39113/ using document root <none> and password file <none>
I20260807 15:10:40.100023 14861 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:40.100075 14861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:40.100201 14861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:40.101248 14861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "0bee4847bcc147118aa20c81a81a5ce8"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "ed18bc1868e744e8ff130ee01f8e199e"
server_key_iv: "dc356ead14d15171bb59392ef4ad3781"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.101603 14861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "0bee4847bcc147118aa20c81a81a5ce8"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "ed18bc1868e744e8ff130ee01f8e199e"
server_key_iv: "dc356ead14d15171bb59392ef4ad3781"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.102829 14861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:10:40.103581 14884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.103760 14861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:40.103847 14861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "0bee4847bcc147118aa20c81a81a5ce8"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "ed18bc1868e744e8ff130ee01f8e199e"
server_key_iv: "dc356ead14d15171bb59392ef4ad3781"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.103910 14861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:40.124704 14861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:40.125030 14861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:40.125166 14861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:40.129160 14861 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:43207
I20260807 15:10:40.129194 14962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:43207 every 8 connection(s)
I20260807 15:10:40.129523 14861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:40.130491 14963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:40.132778 14963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8: Bootstrap starting.
I20260807 15:10:40.133307 14963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:40.133538 14963 log.cc:824] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:40.134152 14963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8: No bootstrap required, opened a new log
I20260807 15:10:40.135826 14963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bee4847bcc147118aa20c81a81a5ce8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43207 } }
I20260807 15:10:40.135984 14963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:40.136020 14963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bee4847bcc147118aa20c81a81a5ce8, State: Initialized, Role: FOLLOWER
I20260807 15:10:40.136176 14963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bee4847bcc147118aa20c81a81a5ce8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43207 } }
I20260807 15:10:40.136256 14963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:40.136286 14963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:40.136335 14963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:40.136930 14963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bee4847bcc147118aa20c81a81a5ce8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43207 } }
I20260807 15:10:40.137024 14963 leader_election.cc:302] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bee4847bcc147118aa20c81a81a5ce8; no voters:
I20260807 15:10:40.137204 14963 leader_election.cc:288] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:40.137252 14967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:40.137276 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 14861
I20260807 15:10:40.137377 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:40.137382 14967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [term 1 LEADER]: Becoming Leader. State: Replica: 0bee4847bcc147118aa20c81a81a5ce8, State: Running, Role: LEADER
I20260807 15:10:40.137466 14963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:40.137477 14967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bee4847bcc147118aa20c81a81a5ce8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43207 } }
I20260807 15:10:40.137576 18948 external_mini_cluster.cc:1428] Setting key c732963242cd6ec2d53924ca35a433b4
I20260807 15:10:40.138178 14968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bee4847bcc147118aa20c81a81a5ce8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bee4847bcc147118aa20c81a81a5ce8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43207 } } }
I20260807 15:10:40.138264 14968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:40.138475 14969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bee4847bcc147118aa20c81a81a5ce8. Latest consensus state: current_term: 1 leader_uuid: "0bee4847bcc147118aa20c81a81a5ce8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bee4847bcc147118aa20c81a81a5ce8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 43207 } } }
I20260807 15:10:40.138868 14969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:40.139002 14975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:40.139487 14975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:40.143265 14975 catalog_manager.cc:1347] Generated new cluster ID: ac1838a26b04471f88960d37d9e76c83
I20260807 15:10:40.143335 14975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:40.143524 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.138144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53976 (local address 127.18.129.62:43207)
0807 15:10:40.138344 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.138352 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.138382 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:10:40.138499 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.138507 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.138575 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.138741 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:40.139147 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.140640 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.141384 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.141991 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.142126 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.142707 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:40.142711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:40.142712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:40.142939 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:40.142983 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:40.142998 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:40.143065 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.143163 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.143245 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260807 15:10:40.149251 14975 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:40.149720 14975 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:40.163812 14975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8: Generated new TSK 0
I20260807 15:10:40.163985 14975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:40.172783 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43207
--builtin_ntp_servers=127.18.129.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:40.254689 14999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:40.254907 14999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:40.254930 14999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:40.256052 14999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:40.256091 14999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:40.256115 14999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:10:40.257463 14999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:40.257704 14999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:40.258030 14999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:40.263931 15015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:40.263943 15020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:40.263968 14999 server_base.cc:1034] running on GCE node
W20260807 15:10:40.263964 15013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:40.264293 14999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:40.264504 14999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:40.265637 14999 hybrid_clock.cc:648] HybridClock initialized: now 1786115440265629 us; error 27 us; skew 500 ppm
I20260807 15:10:40.266934 14999 webserver.cc:460] Webserver started at http://127.18.129.1:35989/ using document root <none> and password file <none>
I20260807 15:10:40.267187 14999 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:40.267239 14999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:40.267370 14999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:40.268504 14999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "39465f9384c64fe2ad1cd8643a0790d5"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "b710d38f0f8f79ed14478d025feb6fcf"
server_key_iv: "f5879e2d507ddc838bfdb0e5b7ea3d8a"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.268849 14999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "39465f9384c64fe2ad1cd8643a0790d5"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "b710d38f0f8f79ed14478d025feb6fcf"
server_key_iv: "f5879e2d507ddc838bfdb0e5b7ea3d8a"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.270219 14999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:10:40.270977 15026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.271138 14999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:10:40.271227 14999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "39465f9384c64fe2ad1cd8643a0790d5"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "b710d38f0f8f79ed14478d025feb6fcf"
server_key_iv: "f5879e2d507ddc838bfdb0e5b7ea3d8a"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.271282 14999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:40.284307 14999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:40.284585 14999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:40.284687 14999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:40.284871 14999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:40.285151 14999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:40.285183 14999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.285209 14999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:40.285224 14999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.290661 14999 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:34349
I20260807 15:10:40.290747 15196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:34349 every 8 connection(s)
I20260807 15:10:40.291052 14999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:40.296195 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.291766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:42467 (local address 127.18.129.62:43207)
0807 15:10:40.291821 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.291823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.292055 (+ 232us) server_negotiation.cc:407] Connection header received
0807 15:10:40.292626 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.292627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.292640 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.292687 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:40.293313 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.293806 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.294839 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.295050 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.295117 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.295916 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:40.295918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:40.295919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:40.295924 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:40.295940 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:40.295944 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:40.295989 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.296068 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.296111 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:10:40.296377 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.291744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43207 (local address 127.18.129.1:42467)
0807 15:10:40.291926 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.291960 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:40.292544 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:40.292748 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:40.292751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:40.292760 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:40.293254 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.293258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.293920 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.293921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.294728 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.294731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.295134 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.295135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.295263 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.295775 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:40.295775 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:40.295804 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:40.295829 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:40.295996 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:40.296001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:40.296004 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:40.296066 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 15:10:40.296171 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260807 15:10:40.296813 15197 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43207
I20260807 15:10:40.296897 15197 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:40.297075 15197 heartbeater.cc:511] Master 127.18.129.62:43207 requested a full tablet report, sending...
I20260807 15:10:40.297560 14912 ts_manager.cc:194] Registered new tserver with Master: 39465f9384c64fe2ad1cd8643a0790d5 (127.18.129.1:34349)
I20260807 15:10:40.298177 14912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.1:42467
I20260807 15:10:40.298563 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 14999
I20260807 15:10:40.298687 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:40.298926 18948 external_mini_cluster.cc:1428] Setting key 9d3af9a525a553c73e6da72875c145e5
I20260807 15:10:40.299757 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43207
--builtin_ntp_servers=127.18.129.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:40.381433 15201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:40.381629 15201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:40.381659 15201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:40.382745 15201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:40.382783 15201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:40.382814 15201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:10:40.384164 15201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:40.384397 15201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:40.384663 15201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:10:40.389599 15201 server_base.cc:1034] running on GCE node
W20260807 15:10:40.389523 15212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:40.389575 15215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:40.389530 15207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:40.389900 15201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:40.390151 15201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:40.391290 15201 hybrid_clock.cc:648] HybridClock initialized: now 1786115440391275 us; error 26 us; skew 500 ppm
I20260807 15:10:40.392490 15201 webserver.cc:460] Webserver started at http://127.18.129.2:35295/ using document root <none> and password file <none>
I20260807 15:10:40.392740 15201 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:40.392788 15201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:40.392912 15201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:40.393919 15201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "0c1c14e8cfd540048d8aeea2153a7038"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "e093739b0620bbc0dcf14fbf7d3945a5"
server_key_iv: "f7e01c524c967bd871f97121de1d92f7"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.394251 15201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "0c1c14e8cfd540048d8aeea2153a7038"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "e093739b0620bbc0dcf14fbf7d3945a5"
server_key_iv: "f7e01c524c967bd871f97121de1d92f7"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.395810 15201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:40.396616 15227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.396775 15201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:40.396862 15201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "0c1c14e8cfd540048d8aeea2153a7038"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "e093739b0620bbc0dcf14fbf7d3945a5"
server_key_iv: "f7e01c524c967bd871f97121de1d92f7"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.396930 15201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:40.403278 15201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:40.403538 15201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:40.403659 15201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:40.403846 15201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:40.404138 15201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:40.404172 15201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.404198 15201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:40.404215 15201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.409823 15201 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:33495
I20260807 15:10:40.409942 15388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:33495 every 8 connection(s)
I20260807 15:10:40.410254 15201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:40.414813 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 15201
I20260807 15:10:40.414950 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:40.415162 18948 external_mini_cluster.cc:1428] Setting key cab959b12c0a91eaf6db659557136f8f
I20260807 15:10:40.415313 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.410946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:35467 (local address 127.18.129.62:43207)
0807 15:10:40.411000 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.411002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.411235 (+ 233us) server_negotiation.cc:407] Connection header received
0807 15:10:40.411776 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.411778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.411790 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.411838 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:40.412502 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.412998 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.413939 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.414129 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.414183 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.415028 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:40.415030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:40.415031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:40.415036 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:40.415056 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:40.415061 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:40.415111 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.415204 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.415235 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:10:40.415537 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.410956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43207 (local address 127.18.129.2:35467)
0807 15:10:40.411125 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.411157 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:40.411672 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:40.411914 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:40.411918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:40.411929 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:40.412405 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.412410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.413075 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.413076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.413851 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.413853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.414196 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.414197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.414325 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.414831 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:40.414831 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:40.414890 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:40.414919 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:40.415131 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:40.415137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:40.415140 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:40.415193 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:10:40.415303 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260807 15:10:40.415951 15391 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43207
I20260807 15:10:40.416038 15391 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:40.416067 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43207
--builtin_ntp_servers=127.18.129.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:10:40.416229 15391 heartbeater.cc:511] Master 127.18.129.62:43207 requested a full tablet report, sending...
I20260807 15:10:40.416590 14912 ts_manager.cc:194] Registered new tserver with Master: 0c1c14e8cfd540048d8aeea2153a7038 (127.18.129.2:33495)
I20260807 15:10:40.416915 14912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.2:35467
W20260807 15:10:40.497473 15398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:40.497674 15398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:40.497704 15398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:40.498819 15398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:40.498878 15398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:40.498911 15398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:10:40.500344 15398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:43207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:40.500609 15398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:40.500882 15398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:40.505853 15408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:40.505831 15406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:40.505851 15404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:40.505868 15398 server_base.cc:1034] running on GCE node
I20260807 15:10:40.506250 15398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:40.506459 15398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:40.507598 15398 hybrid_clock.cc:648] HybridClock initialized: now 1786115440507585 us; error 28 us; skew 500 ppm
I20260807 15:10:40.508828 15398 webserver.cc:460] Webserver started at http://127.18.129.3:42477/ using document root <none> and password file <none>
I20260807 15:10:40.509060 15398 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:40.509110 15398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:40.509234 15398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:40.510286 15398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "a23f4d34941541f3a689383b7286014f"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "9f8a215a2147075423a9f70fd671ff40"
server_key_iv: "191409ab11e512ccba151b3ae1349ca7"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.510617 15398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "a23f4d34941541f3a689383b7286014f"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "9f8a215a2147075423a9f70fd671ff40"
server_key_iv: "191409ab11e512ccba151b3ae1349ca7"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.511895 15398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:10:40.512535 15414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.512691 15398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:10:40.512774 15398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "a23f4d34941541f3a689383b7286014f"
format_stamp: "Formatted at 2026-08-07 15:10:40 on dist-test-slave-5bn2"
server_key: "9f8a215a2147075423a9f70fd671ff40"
server_key_iv: "191409ab11e512ccba151b3ae1349ca7"
server_key_version: "encryptionkey@0"
I20260807 15:10:40.512838 15398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:40.520601 15398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:40.520840 15398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:40.520954 15398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:40.521140 15398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:40.521433 15398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:40.521472 15398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.521498 15398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:40.521515 15398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:40.526991 15398 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:44687
I20260807 15:10:40.527045 15586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:44687 every 8 connection(s)
I20260807 15:10:40.527343 15398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:40.531487 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 15398
I20260807 15:10:40.531610 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:40.531821 18948 external_mini_cluster.cc:1428] Setting key b5a00b700b6d2d7e0983dd25fc5bd56a
I20260807 15:10:40.532361 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.527998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:39225 (local address 127.18.129.62:43207)
0807 15:10:40.528045 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.528047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.528264 (+ 217us) server_negotiation.cc:407] Connection header received
0807 15:10:40.528705 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.528706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.528716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.528766 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:40.529372 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.529860 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.530785 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.531017 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.531069 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.532017 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:40.532019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:40.532019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:40.532025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:40.532042 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:40.532047 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:40.532092 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.532246 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.532278 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:10:40.532565 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.527975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:43207 (local address 127.18.129.3:39225)
0807 15:10:40.528150 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.528182 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:40.528629 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:40.528816 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:40.528819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:40.528826 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:40.529311 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.529315 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.529927 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.529928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.530717 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.530719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.531113 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.531115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.531259 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.531855 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:40.531855 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:40.531894 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:40.531921 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:40.532127 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:40.532159 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:40.532163 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:40.532242 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:10:40.532340 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260807 15:10:40.533035 15588 heartbeater.cc:347] Connected to a master server at 127.18.129.62:43207
I20260807 15:10:40.533123 15588 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:40.533265 15588 heartbeater.cc:511] Master 127.18.129.62:43207 requested a full tablet report, sending...
I20260807 15:10:40.533560 14912 ts_manager.cc:194] Registered new tserver with Master: a23f4d34941541f3a689383b7286014f (127.18.129.3:44687)
I20260807 15:10:40.533902 14912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.3:39225
I20260807 15:10:40.543471 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:10:40.547345 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.544787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.18.129.62:43207)
0807 15:10:40.544857 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.544859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.544920 (+ 61us) server_negotiation.cc:407] Connection header received
0807 15:10:40.544971 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.544972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.544982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.545018 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:40.545156 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.545618 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.546274 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.546443 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.546485 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.547107 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:40.547108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:40.547108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:40.547113 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:40.547127 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:40.547131 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:40.547205 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.547261 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.547271 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:10:40.548700 14912 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:40.550230 14912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:40.556116 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.552139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34349 (local address 127.0.0.1:46956)
0807 15:10:40.552454 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.552488 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:40.552554 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:40.552775 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:40.552778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:40.552780 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:40.552877 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.552879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.554253 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.554254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.555072 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.555075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.555827 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.555827 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.555906 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.555920 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:10:40.555981 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:10:40.556032 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":16,"thread_start_us":98,"threads_started":1}
I20260807 15:10:40.556128 15603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.552246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46956 (local address 127.18.129.1:34349)
0807 15:10:40.552586 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.552591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.552603 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:10:40.552645 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.552647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.552662 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.552767 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:40.552971 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.554194 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.555169 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.555746 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.555811 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.555902 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.556009 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.556048 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":95,"threads_started":1}
I20260807 15:10:40.556537 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.552130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:44687 (local address 127.0.0.1:38396)
0807 15:10:40.552452 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.552490 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:40.552564 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:40.552775 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:40.552778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:40.552783 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:40.552876 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.552879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.554233 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.554233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.555305 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.555308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.556317 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.556318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.556378 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.556382 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:40.556430 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:10:40.556470 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":17,"thread_start_us":56,"threads_started":1}
I20260807 15:10:40.556730 15598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.552335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:33495 (local address 127.0.0.1:41798)
0807 15:10:40.552855 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.552864 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:40.552913 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:40.553062 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:40.553063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:40.553064 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:40.553107 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.553109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.554060 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.554061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.555309 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.555311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.556500 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.556501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.556563 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.556568 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:10:40.556623 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:10:40.556662 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":71,"threads_started":1}
I20260807 15:10:40.556854 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.552218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.18.129.3:44687)
0807 15:10:40.552379 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.552383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.552637 (+ 254us) server_negotiation.cc:407] Connection header received
0807 15:10:40.552678 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.552680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.552696 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.552767 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:40.552971 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.554133 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.555404 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.556123 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.556190 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.556282 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.556719 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.556766 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260807 15:10:40.556857 15100 tablet_service.cc:1467] Processing CreateTablet for tablet b2b59160407441489e07d72b7623a9d0 (DEFAULT_TABLE table=test-table [id=158be3fc7d6844b39bc896e09c30b7bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:40.557001 15606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.552554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.18.129.2:33495)
0807 15:10:40.552716 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.552720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.552926 (+ 206us) server_negotiation.cc:407] Connection header received
0807 15:10:40.552973 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.552975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.552987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.553061 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:40.553183 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.553976 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.555404 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.556123 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.556493 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.556739 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.556763 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.556925 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260807 15:10:40.557250 15100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b59160407441489e07d72b7623a9d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:40.557430 15485 tablet_service.cc:1467] Processing CreateTablet for tablet b2b59160407441489e07d72b7623a9d0 (DEFAULT_TABLE table=test-table [id=158be3fc7d6844b39bc896e09c30b7bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:40.557732 15485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b59160407441489e07d72b7623a9d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:40.560046 15607 tablet_bootstrap.cc:492] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5: Bootstrap starting.
I20260807 15:10:40.560209 15608 tablet_bootstrap.cc:492] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: Bootstrap starting.
I20260807 15:10:40.560672 15290 tablet_service.cc:1467] Processing CreateTablet for tablet b2b59160407441489e07d72b7623a9d0 (DEFAULT_TABLE table=test-table [id=158be3fc7d6844b39bc896e09c30b7bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:40.560840 15607 tablet_bootstrap.cc:654] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:40.560840 15608 tablet_bootstrap.cc:654] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:40.561030 15290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2b59160407441489e07d72b7623a9d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:40.561167 15608 log.cc:824] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:40.561167 15607 log.cc:824] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:40.561906 15608 tablet_bootstrap.cc:492] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: No bootstrap required, opened a new log
I20260807 15:10:40.561916 15607 tablet_bootstrap.cc:492] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5: No bootstrap required, opened a new log
I20260807 15:10:40.561980 15607 ts_tablet_manager.cc:1397] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:40.561980 15608 ts_tablet_manager.cc:1397] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:40.563874 15611 tablet_bootstrap.cc:492] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Bootstrap starting.
I20260807 15:10:40.564443 15611 tablet_bootstrap.cc:654] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:40.564440 15607 raft_consensus.cc:348] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.564415 15608 raft_consensus.cc:348] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.564586 15607 raft_consensus.cc:374] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:40.564587 15608 raft_consensus.cc:374] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:40.564631 15607 raft_consensus.cc:729] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39465f9384c64fe2ad1cd8643a0790d5, State: Initialized, Role: FOLLOWER
I20260807 15:10:40.564631 15608 raft_consensus.cc:729] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23f4d34941541f3a689383b7286014f, State: Initialized, Role: FOLLOWER
I20260807 15:10:40.564767 15611 log.cc:824] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:40.564761 15608 consensus_queue.cc:260] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.564761 15607 consensus_queue.cc:260] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.565009 15607 ts_tablet_manager.cc:1428] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:40.565025 15608 ts_tablet_manager.cc:1428] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:10:40.565146 15197 heartbeater.cc:503] Master 127.18.129.62:43207 was elected leader, sending a full tablet report...
I20260807 15:10:40.565403 15588 heartbeater.cc:503] Master 127.18.129.62:43207 was elected leader, sending a full tablet report...
I20260807 15:10:40.565686 15611 tablet_bootstrap.cc:492] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: No bootstrap required, opened a new log
I20260807 15:10:40.565752 15611 ts_tablet_manager.cc:1397] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:40.567718 15611 raft_consensus.cc:348] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.567843 15611 raft_consensus.cc:374] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:40.567870 15611 raft_consensus.cc:729] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c1c14e8cfd540048d8aeea2153a7038, State: Initialized, Role: FOLLOWER
I20260807 15:10:40.567981 15611 consensus_queue.cc:260] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.568249 15611 ts_tablet_manager.cc:1428] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:10:40.568277 15391 heartbeater.cc:503] Master 127.18.129.62:43207 was elected leader, sending a full tablet report...
I20260807 15:10:40.644546 15615 raft_consensus.cc:482] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:40.644688 15615 raft_consensus.cc:504] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.645033 15615 leader_election.cc:288] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39465f9384c64fe2ad1cd8643a0790d5 (127.18.129.1:34349), a23f4d34941541f3a689383b7286014f (127.18.129.3:44687)
I20260807 15:10:40.647626 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.645197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:44687 (local address 127.18.129.2:39567)
0807 15:10:40.645246 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.645258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:40.645298 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:40.645426 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:40.645427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:40.645428 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:40.645496 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.645498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.646080 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.646081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.646827 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.646829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.647375 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.647376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.647452 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.647455 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:40.647499 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:10:40.647544 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:10:40.647619 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.645266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:39567 (local address 127.18.129.3:44687)
0807 15:10:40.645321 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.645323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.645332 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:10:40.645362 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.645363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.645370 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.645416 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:40.645553 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.646011 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.646939 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.647305 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.647367 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.647376 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.647515 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.647552 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:10:40.647853 15603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.645298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:35955 (local address 127.18.129.1:34349)
0807 15:10:40.645345 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.645347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.645488 (+ 141us) server_negotiation.cc:407] Connection header received
0807 15:10:40.645556 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.645558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.645569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.645624 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:40.645756 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.646225 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.647167 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.647519 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.647581 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.647588 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.647741 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.647785 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:10:40.647939 15518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b59160407441489e07d72b7623a9d0" candidate_uuid: "0c1c14e8cfd540048d8aeea2153a7038" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23f4d34941541f3a689383b7286014f" is_pre_election: true
I20260807 15:10:40.648083 15616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.645200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34349 (local address 127.18.129.2:35955)
0807 15:10:40.645418 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.645429 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:40.645486 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:40.645627 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:40.645628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:40.645629 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:40.645701 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.645703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.646294 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.646295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.647089 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:40.647091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.647586 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.647587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:40.647654 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.647656 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:10:40.647966 (+ 310us) client_negotiation.cc:240] Negotiation successful
0807 15:10:40.648007 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":60,"thread_start_us":111,"threads_started":1}
I20260807 15:10:40.648102 15518 raft_consensus.cc:2454] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c1c14e8cfd540048d8aeea2153a7038 in term 0.
I20260807 15:10:40.648341 15234 leader_election.cc:302] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c1c14e8cfd540048d8aeea2153a7038, a23f4d34941541f3a689383b7286014f; no voters:
I20260807 15:10:40.648396 15125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b59160407441489e07d72b7623a9d0" candidate_uuid: "0c1c14e8cfd540048d8aeea2153a7038" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39465f9384c64fe2ad1cd8643a0790d5" is_pre_election: true
I20260807 15:10:40.648541 15125 raft_consensus.cc:2454] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c1c14e8cfd540048d8aeea2153a7038 in term 0.
I20260807 15:10:40.648550 15615 raft_consensus.cc:2781] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:40.648619 15615 raft_consensus.cc:482] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:40.648723 15615 raft_consensus.cc:3037] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:40.649435 15615 raft_consensus.cc:504] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.649559 15615 leader_election.cc:288] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [CANDIDATE]: Term 1 election: Requested vote from peers 39465f9384c64fe2ad1cd8643a0790d5 (127.18.129.1:34349), a23f4d34941541f3a689383b7286014f (127.18.129.3:44687)
I20260807 15:10:40.649724 15125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b59160407441489e07d72b7623a9d0" candidate_uuid: "0c1c14e8cfd540048d8aeea2153a7038" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39465f9384c64fe2ad1cd8643a0790d5"
I20260807 15:10:40.649799 15125 raft_consensus.cc:3037] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:40.650543 15125 raft_consensus.cc:2454] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c1c14e8cfd540048d8aeea2153a7038 in term 1.
I20260807 15:10:40.650676 15518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2b59160407441489e07d72b7623a9d0" candidate_uuid: "0c1c14e8cfd540048d8aeea2153a7038" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a23f4d34941541f3a689383b7286014f"
I20260807 15:10:40.650753 15230 leader_election.cc:302] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c1c14e8cfd540048d8aeea2153a7038, 39465f9384c64fe2ad1cd8643a0790d5; no voters:
I20260807 15:10:40.650755 15518 raft_consensus.cc:3037] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:40.650883 15615 raft_consensus.cc:2781] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:40.651044 15615 raft_consensus.cc:686] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 1 LEADER]: Becoming Leader. State: Replica: 0c1c14e8cfd540048d8aeea2153a7038, State: Running, Role: LEADER
I20260807 15:10:40.651149 15615 consensus_queue.cc:237] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } }
I20260807 15:10:40.651705 15518 raft_consensus.cc:2454] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c1c14e8cfd540048d8aeea2153a7038 in term 1.
I20260807 15:10:40.651998 14911 catalog_manager.cc:5495] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c1c14e8cfd540048d8aeea2153a7038 (127.18.129.2). New cstate: current_term: 1 leader_uuid: "0c1c14e8cfd540048d8aeea2153a7038" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c1c14e8cfd540048d8aeea2153a7038" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 33495 } health_report { overall_health: HEALTHY } } }
W20260807 15:10:40.660774 15395 tablet.cc:2369] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:40.671917 15606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.669205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.18.129.2:33495)
0807 15:10:40.669267 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:40.669269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:40.669280 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:10:40.669319 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:40.669320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:40.669401 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:40.669480 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:40.669690 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.670450 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.671227 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:40.671478 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:40.671541 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:40.671645 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:40.671733 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:40.671804 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:40.671825 (+ 21us) server_negotiation.cc:299] Negotiation successful
0807 15:10:40.671835 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:10:40.673684 15518 raft_consensus.cc:1261] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 1 FOLLOWER]: Refusing update from remote peer 0c1c14e8cfd540048d8aeea2153a7038: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:40.673993 15125 raft_consensus.cc:1261] T b2b59160407441489e07d72b7623a9d0 P 39465f9384c64fe2ad1cd8643a0790d5 [term 1 FOLLOWER]: Refusing update from remote peer 0c1c14e8cfd540048d8aeea2153a7038: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:40.674024 15615 consensus_queue.cc:1035] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a23f4d34941541f3a689383b7286014f" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:40.674234 15615 consensus_queue.cc:1035] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39465f9384c64fe2ad1cd8643a0790d5" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:40.676932 15626 mvcc.cc:205] Tried to move back new op lower bound from 7315928844996825088 to 7315928844907819008. Current Snapshot: MvccSnapshot[applied={T|T < 7315928844996825088 or (T in {7315928844996825088})}]
I20260807 15:10:40.679170 14912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:40.679258 14912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:40.679983 14912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bee4847bcc147118aa20c81a81a5ce8: Sending DeleteTablet for 3 replicas of tablet b2b59160407441489e07d72b7623a9d0
I20260807 15:10:40.680310 15100 tablet_service.cc:1514] Processing DeleteTablet for tablet b2b59160407441489e07d72b7623a9d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:40 UTC) from {username='slave'} at 127.0.0.1:46956
I20260807 15:10:40.680392 15485 tablet_service.cc:1514] Processing DeleteTablet for tablet b2b59160407441489e07d72b7623a9d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:40 UTC) from {username='slave'} at 127.0.0.1:38396
I20260807 15:10:40.680539 15290 tablet_service.cc:1514] Processing DeleteTablet for tablet b2b59160407441489e07d72b7623a9d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:40 UTC) from {username='slave'} at 127.0.0.1:41798
I20260807 15:10:40.680609 15636 tablet_replica.cc:330] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: stopping tablet replica
I20260807 15:10:40.680714 15636 raft_consensus.cc:2229] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:40.680804 15638 tablet_replica.cc:330] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: stopping tablet replica
I20260807 15:10:40.680821 15636 raft_consensus.cc:2258] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:40.681028 15638 raft_consensus.cc:2229] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:40.681063 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 14999
I20260807 15:10:40.681238 15638 raft_consensus.cc:2258] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:40.687484 15636 ts_tablet_manager.cc:1905] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:40.689666 15638 ts_tablet_manager.cc:1905] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:40.691197 15638 ts_tablet_manager.cc:1918] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:40.691282 15638 log.cc:1196] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/b2b59160407441489e07d72b7623a9d0
I20260807 15:10:40.691543 15638 ts_tablet_manager.cc:1939] T b2b59160407441489e07d72b7623a9d0 P 0c1c14e8cfd540048d8aeea2153a7038: Deleting consensus metadata
I20260807 15:10:40.691959 14888 catalog_manager.cc:4841] TS 0c1c14e8cfd540048d8aeea2153a7038 (127.18.129.2:33495): tablet b2b59160407441489e07d72b7623a9d0 (table test-table [id=158be3fc7d6844b39bc896e09c30b7bd]) successfully deleted
I20260807 15:10:40.692006 15636 ts_tablet_manager.cc:1918] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:40.692075 15636 log.cc:1196] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/b2b59160407441489e07d72b7623a9d0
I20260807 15:10:40.692309 15636 ts_tablet_manager.cc:1939] T b2b59160407441489e07d72b7623a9d0 P a23f4d34941541f3a689383b7286014f: Deleting consensus metadata
I20260807 15:10:40.692848 14892 catalog_manager.cc:4841] TS a23f4d34941541f3a689383b7286014f (127.18.129.3:44687): tablet b2b59160407441489e07d72b7623a9d0 (table test-table [id=158be3fc7d6844b39bc896e09c30b7bd]) successfully deleted
W20260807 15:10:40.694609 14888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:34349 (error 108)
I20260807 15:10:40.694947 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 15201
I20260807 15:10:40.695477 15598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:40.694979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:34349 (local address 127.0.0.1:46970)
0807 15:10:40.695389 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:40.695441 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387}
W20260807 15:10:40.697345 14888 catalog_manager.cc:4584] TS 39465f9384c64fe2ad1cd8643a0790d5 (127.18.129.1:34349): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2b59160407441489e07d72b7623a9d0: Network error: Client connection negotiation failed: client connection to 127.18.129.1:34349: connect: Connection refused (error 111)
I20260807 15:10:40.702706 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 15398
I20260807 15:10:40.711779 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 14861
2026-08-07T15:10:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (739 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:40 dist-test-slave-5bn2 krb5kdc[15645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:40 dist-test-slave-5bn2 krb5kdc[15645](info): set up 2 sockets
Aug 07 15:10:40 dist-test-slave-5bn2 krb5kdc[15645](info): commencing operation
krb5kdc: starting...
W20260807 15:10:41.840909 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:41 dist-test-slave-5bn2 krb5kdc[15645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:10:41.960026 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:40005
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:45591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:40005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:42.040725 15660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:42.040982 15660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:42.041021 15660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:42.042413 15660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:42.042456 15660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:42.042474 15660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:42.042491 15660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:42.042506 15660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:42.043979 15660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:40005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:40005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:42.044250 15660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:42.044456 15660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:42.049332 15673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.049405 15660 server_base.cc:1034] running on GCE node
W20260807 15:10:42.049434 15669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:42.049332 15675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.049817 15660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:42.050163 15660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:42.051316 15660 hybrid_clock.cc:648] HybridClock initialized: now 1786115442051289 us; error 37 us; skew 500 ppm
Aug 07 15:10:42 dist-test-slave-5bn2 krb5kdc[15645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:42.053916 15660 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:10:42.054359 15660 webserver.cc:460] Webserver started at http://127.18.129.62:39461/ using document root <none> and password file <none>
I20260807 15:10:42.054561 15660 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:42.054611 15660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:42.054731 15660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:42.055830 15660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "64c8c532e834452cb0a29494e7b832cd"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "4f76b59c5ce1bea8f829f73304d49d2e"
server_key_iv: "ba918c6a32df52945f41358dff15612b"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.056175 15660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "64c8c532e834452cb0a29494e7b832cd"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "4f76b59c5ce1bea8f829f73304d49d2e"
server_key_iv: "ba918c6a32df52945f41358dff15612b"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.057430 15660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:42.058111 15686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.058261 15660 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:42.058346 15660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "64c8c532e834452cb0a29494e7b832cd"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "4f76b59c5ce1bea8f829f73304d49d2e"
server_key_iv: "ba918c6a32df52945f41358dff15612b"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.058413 15660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:42.073009 15660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:42.075107 15660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:42.075245 15660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:42.079188 15660 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:40005
I20260807 15:10:42.079229 15759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:40005 every 8 connection(s)
I20260807 15:10:42.079556 15660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:42.080411 15761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:42.082727 15761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd: Bootstrap starting.
I20260807 15:10:42.083330 15761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:42.083616 15761 log.cc:824] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:42.084281 15761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd: No bootstrap required, opened a new log
I20260807 15:10:42.084964 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 15660
I20260807 15:10:42.085078 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:42.085300 18948 external_mini_cluster.cc:1428] Setting key 655c9fb676cb9482d203dd192efeb704
I20260807 15:10:42.086355 15761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c8c532e834452cb0a29494e7b832cd" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40005 } }
I20260807 15:10:42.086519 15761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:42.086567 15761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c8c532e834452cb0a29494e7b832cd, State: Initialized, Role: FOLLOWER
I20260807 15:10:42.086720 15761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c8c532e834452cb0a29494e7b832cd" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40005 } }
I20260807 15:10:42.086809 15761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:42.086869 15761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:42.086925 15761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:42.087891 15761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c8c532e834452cb0a29494e7b832cd" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40005 } }
I20260807 15:10:42.087983 15761 leader_election.cc:302] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64c8c532e834452cb0a29494e7b832cd; no voters:
I20260807 15:10:42.088130 15761 leader_election.cc:288] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:42.088173 15767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:42.088305 15767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [term 1 LEADER]: Becoming Leader. State: Replica: 64c8c532e834452cb0a29494e7b832cd, State: Running, Role: LEADER
I20260807 15:10:42.088378 15761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:42.088388 15767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c8c532e834452cb0a29494e7b832cd" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40005 } }
I20260807 15:10:42.088824 15767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64c8c532e834452cb0a29494e7b832cd. Latest consensus state: current_term: 1 leader_uuid: "64c8c532e834452cb0a29494e7b832cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c8c532e834452cb0a29494e7b832cd" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40005 } } }
I20260807 15:10:42.088897 15767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:42.089012 15768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64c8c532e834452cb0a29494e7b832cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c8c532e834452cb0a29494e7b832cd" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 40005 } } }
I20260807 15:10:42.089161 15768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:42.089252 15775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:42.089767 15775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:10:42 dist-test-slave-5bn2 krb5kdc[15645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:42.094920 15775 catalog_manager.cc:1347] Generated new cluster ID: f4f9da2efac84e1a9ad704b55cf80b52
I20260807 15:10:42.094980 15775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:42.096115 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.085809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54382 (local address 127.18.129.62:40005)
0807 15:10:42.086014 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.086021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.086046 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:10:42.086317 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.086322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.086364 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.086472 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:42.087323 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.088361 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.089370 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.089993 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.090138 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.092783 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:42.092788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:42.092788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:42.092800 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:42.094615 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.094942 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.094945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.094945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.095078 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.095276 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.095277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.095277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.095562 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:42.095707 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.095773 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.095902 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260807 15:10:42.103933 15775 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:42.104398 15775 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:42.109429 15775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd: Generated new TSK 0
I20260807 15:10:42.109573 15775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.129.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
Entry for principal kudu/127.18.129.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab.
I20260807 15:10:42.143649 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40005
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:45591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:42.227859 15796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:42.228094 15796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:42.228118 15796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:42.229310 15796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:42.229354 15796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:42.229383 15796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:10:42.231155 15796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.1.keytab
--principal=kudu/127.18.129.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:42.231421 15796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:42.231649 15796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:42.236601 15804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:42.236613 15808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:42.236613 15805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.236816 15796 server_base.cc:1034] running on GCE node
I20260807 15:10:42.237010 15796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:42.237241 15796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:42.238379 15796 hybrid_clock.cc:648] HybridClock initialized: now 1786115442238362 us; error 29 us; skew 500 ppm
Aug 07 15:10:42 dist-test-slave-5bn2 krb5kdc[15645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:42.240852 15796 init.cc:377] Logged in from keytab as kudu/127.18.129.1@KRBTEST.COM (short username kudu)
I20260807 15:10:42.241294 15796 webserver.cc:460] Webserver started at http://127.18.129.1:33723/ using document root <none> and password file <none>
I20260807 15:10:42.241509 15796 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:42.241557 15796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:42.241699 15796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:42.242731 15796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "490d8805c3ff894a79197ed1fbe76e49"
server_key_iv: "3adc7c8ebcb7433ee12122e1da860310"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.243122 15796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "490d8805c3ff894a79197ed1fbe76e49"
server_key_iv: "3adc7c8ebcb7433ee12122e1da860310"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.244393 15796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:42.245052 15818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.245204 15796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:42.245291 15796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "490d8805c3ff894a79197ed1fbe76e49"
server_key_iv: "3adc7c8ebcb7433ee12122e1da860310"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.245352 15796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:42.270951 15796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:42.272801 15796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:42.272948 15796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:42.273183 15796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:42.273496 15796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:42.273532 15796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.273563 15796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:42.273595 15796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.279181 15796 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:38527
I20260807 15:10:42.279235 15984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:38527 every 8 connection(s)
I20260807 15:10:42.279556 15796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:10:42 dist-test-slave-5bn2 krb5kdc[15645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.1@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:42.288201 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.280238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:36557 (local address 127.18.129.62:40005)
0807 15:10:42.280293 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.280295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.280485 (+ 190us) server_negotiation.cc:407] Connection header received
0807 15:10:42.281006 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.281008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.281025 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.281078 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:42.282075 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.282593 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.283623 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.283846 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.283906 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.285958 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:42.285960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:42.285960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:42.285970 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:42.287150 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.287493 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.287493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.287494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.287539 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.287717 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.287717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.287718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.287813 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:42.287858 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.288060 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.288134 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:10:42.288401 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.280191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40005 (local address 127.18.129.1:36557)
0807 15:10:42.280364 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.280402 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:42.280905 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:42.281154 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:42.281159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:42.281738 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:42.281952 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.281958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.282690 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.282691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.283524 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.283527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.283924 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.283925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.284047 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.284499 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:42.284507 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:42.285870 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:42.287225 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:42.287232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:42.287234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:42.287441 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:42.287582 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:42.287583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:42.287583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:42.287676 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:42.287854 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:42.287857 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:42.287968 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 15:10:42.288042 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:10:42.288161 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260807 15:10:42.288887 15986 heartbeater.cc:347] Connected to a master server at 127.18.129.62:40005
I20260807 15:10:42.288986 15986 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:42.289187 15986 heartbeater.cc:511] Master 127.18.129.62:40005 requested a full tablet report, sending...
I20260807 15:10:42.289629 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 15796
I20260807 15:10:42.289729 15711 ts_manager.cc:194] Registered new tserver with Master: 68f7dc2a0bac4fcc88c6cddd29ce03b6 (127.18.129.1:38527)
I20260807 15:10:42.289753 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:42.289952 18948 external_mini_cluster.cc:1428] Setting key 6327a22fe9d5a360533354fbd1cd4463
I20260807 15:10:42.290457 15711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.1@KRBTEST.COM'} at 127.18.129.1:36557
WARNING: no policy specified for kudu/127.18.129.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
Entry for principal kudu/127.18.129.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab.
I20260807 15:10:42.313267 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40005
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:45591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:42.394553 15991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:42.394758 15991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:42.394779 15991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:42.395893 15991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:42.395933 15991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:42.395958 15991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:10:42.397351 15991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.2.keytab
--principal=kudu/127.18.129.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:42.397547 15991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:42.397727 15991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:42.402673 16004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.402784 15991 server_base.cc:1034] running on GCE node
W20260807 15:10:42.402751 16002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:42.402724 16001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.403116 15991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:42.403323 15991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:42.404453 15991 hybrid_clock.cc:648] HybridClock initialized: now 1786115442404438 us; error 33 us; skew 500 ppm
Aug 07 15:10:42 dist-test-slave-5bn2 krb5kdc[15645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:42.406842 15991 init.cc:377] Logged in from keytab as kudu/127.18.129.2@KRBTEST.COM (short username kudu)
I20260807 15:10:42.407297 15991 webserver.cc:460] Webserver started at http://127.18.129.2:35999/ using document root <none> and password file <none>
I20260807 15:10:42.407492 15991 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:42.407547 15991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:42.407670 15991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:42.408717 15991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "3fcb43a4f460473b8120ddc7284cd221"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "386a55e0546288c4273bbe4401beb7b1"
server_key_iv: "6e36d0dea2a7f9c9ac91d66ac9d30f7b"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.409044 15991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "3fcb43a4f460473b8120ddc7284cd221"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "386a55e0546288c4273bbe4401beb7b1"
server_key_iv: "6e36d0dea2a7f9c9ac91d66ac9d30f7b"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.410240 15991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:10:42.410890 16014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.411031 15991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:42.411118 15991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "3fcb43a4f460473b8120ddc7284cd221"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "386a55e0546288c4273bbe4401beb7b1"
server_key_iv: "6e36d0dea2a7f9c9ac91d66ac9d30f7b"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.411185 15991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:42.425078 15991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:42.427162 15991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:42.427299 15991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:42.427515 15991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:42.427809 15991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:42.427843 15991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.427872 15991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:42.427896 15991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.433568 15991 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:45599
I20260807 15:10:42.433622 16179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:45599 every 8 connection(s)
I20260807 15:10:42.433923 15991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:42.437629 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 15991
I20260807 15:10:42.437745 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:42.437937 18948 external_mini_cluster.cc:1428] Setting key 12407fca7e48a2ee0d11946e2b949d9b
Aug 07 15:10:42 dist-test-slave-5bn2 krb5kdc[15645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.2@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:42.443207 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.434606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:32797 (local address 127.18.129.62:40005)
0807 15:10:42.434669 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.434671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.434879 (+ 208us) server_negotiation.cc:407] Connection header received
0807 15:10:42.435413 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.435414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.435424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.435472 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:42.436344 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.436838 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.437774 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.438032 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.438093 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.440460 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:42.440462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:42.440462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:42.440470 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:42.441833 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.442283 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.442284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.442284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.442331 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.442574 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.442574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.442574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.442665 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:42.442708 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.443062 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.443126 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:10:42.443478 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.434581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40005 (local address 127.18.129.2:32797)
0807 15:10:42.434743 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.434779 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:42.435328 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:42.435532 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:42.435536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:42.436086 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:42.436277 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.436282 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.436898 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.436899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.437689 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.437692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.438104 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.438105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.438241 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.438737 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:42.438745 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:42.440348 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:42.441907 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:42.441915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:42.441917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:42.442218 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:42.442381 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:42.442381 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:42.442382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:42.442518 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:42.442714 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:42.442721 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:42.442913 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 15:10:42.443050 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 15:10:42.443237 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260807 15:10:42.443953 16180 heartbeater.cc:347] Connected to a master server at 127.18.129.62:40005
I20260807 15:10:42.444046 16180 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:42.444208 16180 heartbeater.cc:511] Master 127.18.129.62:40005 requested a full tablet report, sending...
I20260807 15:10:42.444569 15711 ts_manager.cc:194] Registered new tserver with Master: 3fcb43a4f460473b8120ddc7284cd221 (127.18.129.2:45599)
I20260807 15:10:42.445057 15711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.2@KRBTEST.COM'} at 127.18.129.2:32797
WARNING: no policy specified for kudu/127.18.129.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
Entry for principal kudu/127.18.129.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab.
I20260807 15:10:42.463863 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40005
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:45591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:42.544893 16189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:42.545101 16189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:42.545123 16189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:42.546191 16189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:42.546222 16189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:42.546245 16189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:10:42.547665 16189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:45591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.3.keytab
--principal=kudu/127.18.129.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:40005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:42.547845 16189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:42.548024 16189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:42.553063 16197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:42.553063 16195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.553217 16189 server_base.cc:1034] running on GCE node
W20260807 15:10:42.553082 16199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.553478 16189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:42.553798 16189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:42.555038 16189 hybrid_clock.cc:648] HybridClock initialized: now 1786115442555022 us; error 25 us; skew 500 ppm
Aug 07 15:10:42 dist-test-slave-5bn2 krb5kdc[15645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:42.557603 16189 init.cc:377] Logged in from keytab as kudu/127.18.129.3@KRBTEST.COM (short username kudu)
I20260807 15:10:42.558050 16189 webserver.cc:460] Webserver started at http://127.18.129.3:36779/ using document root <none> and password file <none>
I20260807 15:10:42.558266 16189 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:42.558307 16189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:42.558408 16189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:42.559479 16189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "6bf9a1109c884772b4678f3542733def"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "afa878ac640e21550994ab38a94b6ec7"
server_key_iv: "9874eda53c204cb303123bb4071fe07f"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.559791 16189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "6bf9a1109c884772b4678f3542733def"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "afa878ac640e21550994ab38a94b6ec7"
server_key_iv: "9874eda53c204cb303123bb4071fe07f"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.561080 16189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:42.561800 16212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.561959 16189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:42.562047 16189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "6bf9a1109c884772b4678f3542733def"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "afa878ac640e21550994ab38a94b6ec7"
server_key_iv: "9874eda53c204cb303123bb4071fe07f"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.562119 16189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:42.578825 16189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:42.580966 16189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:42.581117 16189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:42.581358 16189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:42.581682 16189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:42.581718 16189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.581749 16189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:42.581779 16189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.587850 16189 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:43165
I20260807 15:10:42.587944 16373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:43165 every 8 connection(s)
I20260807 15:10:42.588289 16189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:42.588796 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 16189
I20260807 15:10:42.588949 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:42.589205 18948 external_mini_cluster.cc:1428] Setting key 858252864e240b7f23be8112836144ed
Aug 07 15:10:42 dist-test-slave-5bn2 krb5kdc[15645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.3@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:10:42.597443 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.589087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:37357 (local address 127.18.129.62:40005)
0807 15:10:42.589137 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.589139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.589375 (+ 236us) server_negotiation.cc:407] Connection header received
0807 15:10:42.589992 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.589993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.590006 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.590053 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:42.591138 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.591713 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.592671 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.592867 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.592926 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.595119 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:42.595120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:42.595121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:42.595128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:42.596236 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.596592 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.596593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.596594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.596651 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.596842 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.596843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.596844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.596953 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:42.597002 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.597274 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.597374 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:10:42.597517 16376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.589083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:40005 (local address 127.18.129.3:37357)
0807 15:10:42.589244 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.589290 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:42.589889 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:42.590169 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:42.590174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:42.590765 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:42.591029 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.591034 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.591791 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.591791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.592594 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.592596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.592933 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.592934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.593032 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.593531 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:42.593539 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:42.595039 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:42.596302 (+ 1263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:42.596310 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:42.596312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:42.596530 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:42.596699 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:42.596700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:42.596700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:42.596795 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:42.596996 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:42.596999 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:42.597129 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:10:42.597208 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:10:42.597338 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260807 15:10:42.598126 16374 heartbeater.cc:347] Connected to a master server at 127.18.129.62:40005
I20260807 15:10:42.598239 16374 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:42.598484 16374 heartbeater.cc:511] Master 127.18.129.62:40005 requested a full tablet report, sending...
I20260807 15:10:42.598920 15711 ts_manager.cc:194] Registered new tserver with Master: 6bf9a1109c884772b4678f3542733def (127.18.129.3:43165)
I20260807 15:10:42.599306 15711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.129.3@KRBTEST.COM'} at 127.18.129.3:37357
I20260807 15:10:42.601033 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:10:42.607883 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.602319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.18.129.62:40005)
0807 15:10:42.602361 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.602364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.602455 (+ 91us) server_negotiation.cc:407] Connection header received
0807 15:10:42.602505 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.602506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.602514 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.602561 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:42.603130 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.603656 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.604320 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.604548 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.604598 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.605718 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:42.605720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:42.605720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:42.605728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:42.606932 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.607242 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.607242 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.607243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.607298 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:42.607433 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:42.607433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:42.607433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:42.607592 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:42.607661 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.607757 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.607814 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:10:42.609428 15711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:42.611070 15711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:42.617957 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.613765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.18.129.3:43165)
0807 15:10:42.614148 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.614152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.614161 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:10:42.614198 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.614200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.614214 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.614465 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:42.614531 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.615332 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.616780 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.617534 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.617609 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.617673 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.617828 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.617874 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":96,"threads_started":1}
I20260807 15:10:42.617945 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.613465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:43165 (local address 127.0.0.1:40064)
0807 15:10:42.613633 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.613675 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:42.613744 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:42.614301 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:42.614301 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:42.614303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:42.614381 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.614384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.615431 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.615432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.616682 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.616685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.617623 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.617624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.617704 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.617722 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:10:42.617790 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:10:42.617853 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260807 15:10:42.618098 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.613465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:38527 (local address 127.0.0.1:46840)
0807 15:10:42.613960 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.613971 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:42.614030 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:42.614284 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:42.614287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:42.614289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:42.614461 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.614463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.615785 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.615787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.616857 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.616860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.617850 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.617851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.617914 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.617918 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:42.617963 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:10:42.618020 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":30,"thread_start_us":110,"threads_started":1}
I20260807 15:10:42.618398 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.613765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.18.129.1:38527)
0807 15:10:42.613943 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.613947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.614039 (+ 92us) server_negotiation.cc:407] Connection header received
0807 15:10:42.614092 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.614095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.614112 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.614184 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:42.614541 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.615678 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.616993 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.617704 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.617764 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.617817 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.618276 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.618320 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260807 15:10:42.619108 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.613466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:45599 (local address 127.0.0.1:48126)
0807 15:10:42.613946 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.613956 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:42.614007 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:42.614283 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:42.614287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:42.614292 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:42.614366 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.614368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.616977 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.616977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.617662 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.617664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.618880 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.618881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.618944 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.618948 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:42.618995 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:10:42.619040 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":3,"thread_start_us":70,"threads_started":1}
I20260807 15:10:42.619118 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.613664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.18.129.2:45599)
0807 15:10:42.613895 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.613900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.614013 (+ 113us) server_negotiation.cc:407] Connection header received
0807 15:10:42.614064 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.614066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.614082 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.614151 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:42.614464 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.615674 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.617734 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.618791 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.618872 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.618931 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.619007 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.619046 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260807 15:10:42.619043 15890 tablet_service.cc:1467] Processing CreateTablet for tablet faf8e391dacd4bfabfeff85574a8baee (DEFAULT_TABLE table=test-table [id=9f259625517348ebb1635ce506428fd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:42.619041 16279 tablet_service.cc:1467] Processing CreateTablet for tablet faf8e391dacd4bfabfeff85574a8baee (DEFAULT_TABLE table=test-table [id=9f259625517348ebb1635ce506428fd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:42.619400 16279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf8e391dacd4bfabfeff85574a8baee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:42.619408 15890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf8e391dacd4bfabfeff85574a8baee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:42.619714 16086 tablet_service.cc:1467] Processing CreateTablet for tablet faf8e391dacd4bfabfeff85574a8baee (DEFAULT_TABLE table=test-table [id=9f259625517348ebb1635ce506428fd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:42.619980 16086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf8e391dacd4bfabfeff85574a8baee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:42.622728 16398 tablet_bootstrap.cc:492] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: Bootstrap starting.
I20260807 15:10:42.623034 16399 tablet_bootstrap.cc:492] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: Bootstrap starting.
I20260807 15:10:42.623368 16398 tablet_bootstrap.cc:654] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:42.623657 16399 tablet_bootstrap.cc:654] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:42.623884 16398 log.cc:824] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:42.624166 16399 log.cc:824] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:42.624496 16401 tablet_bootstrap.cc:492] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6: Bootstrap starting.
I20260807 15:10:42.625029 16398 tablet_bootstrap.cc:492] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: No bootstrap required, opened a new log
I20260807 15:10:42.625058 16401 tablet_bootstrap.cc:654] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:42.625108 16398 ts_tablet_manager.cc:1397] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:42.625553 16401 log.cc:824] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:42.626125 16399 tablet_bootstrap.cc:492] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: No bootstrap required, opened a new log
I20260807 15:10:42.626189 16399 ts_tablet_manager.cc:1397] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:10:42.626341 16401 tablet_bootstrap.cc:492] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6: No bootstrap required, opened a new log
I20260807 15:10:42.626418 16401 ts_tablet_manager.cc:1397] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:42.627038 16398 raft_consensus.cc:348] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.627240 16398 raft_consensus.cc:374] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:42.627290 16398 raft_consensus.cc:729] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcb43a4f460473b8120ddc7284cd221, State: Initialized, Role: FOLLOWER
I20260807 15:10:42.627424 16398 consensus_queue.cc:260] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.627749 16398 ts_tablet_manager.cc:1428] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:42.627753 16180 heartbeater.cc:503] Master 127.18.129.62:40005 was elected leader, sending a full tablet report...
I20260807 15:10:42.628080 16399 raft_consensus.cc:348] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.628250 16399 raft_consensus.cc:374] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:42.628289 16399 raft_consensus.cc:729] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bf9a1109c884772b4678f3542733def, State: Initialized, Role: FOLLOWER
I20260807 15:10:42.628404 16399 consensus_queue.cc:260] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.628409 16401 raft_consensus.cc:348] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.628598 16401 raft_consensus.cc:374] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:42.628646 16401 raft_consensus.cc:729] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f7dc2a0bac4fcc88c6cddd29ce03b6, State: Initialized, Role: FOLLOWER
I20260807 15:10:42.628686 16399 ts_tablet_manager.cc:1428] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:10:42.628760 16401 consensus_queue.cc:260] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.628865 16374 heartbeater.cc:503] Master 127.18.129.62:40005 was elected leader, sending a full tablet report...
I20260807 15:10:42.629046 15986 heartbeater.cc:503] Master 127.18.129.62:40005 was elected leader, sending a full tablet report...
I20260807 15:10:42.629218 16401 ts_tablet_manager.cc:1428] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:42.635681 16409 raft_consensus.cc:482] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:42.635788 16409 raft_consensus.cc:504] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.636147 16409 leader_election.cc:288] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f7dc2a0bac4fcc88c6cddd29ce03b6 (127.18.129.1:38527), 3fcb43a4f460473b8120ddc7284cd221 (127.18.129.2:45599)
I20260807 15:10:42.640084 16376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.636297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:38527 (local address 127.18.129.3:45173)
0807 15:10:42.636351 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.636363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:42.636402 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:42.636560 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:42.636561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:42.636562 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:42.636650 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.636652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.637604 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.637605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.639010 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.639014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.639779 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.639781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.639873 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.639877 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:42.639942 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:10:42.639982 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":13}
I20260807 15:10:42.640105 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.636445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:57975 (local address 127.18.129.2:45599)
0807 15:10:42.636494 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.636495 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.636536 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:10:42.636599 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.636600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.636609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.636644 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:42.636778 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.637502 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.639097 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.639686 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.639761 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.639770 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.639981 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.640033 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:10:42.640105 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.636377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:45173 (local address 127.18.129.1:38527)
0807 15:10:42.636430 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.636432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.636442 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:10:42.636469 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.636470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.636480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.636547 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:42.636714 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.637524 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.639096 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.639685 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.639761 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.639770 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.639981 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.640033 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:10:42.640106 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.636310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:45599 (local address 127.18.129.3:57975)
0807 15:10:42.636493 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.636500 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:42.636545 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:42.636661 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:42.636661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:42.636663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:42.636723 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.636724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.637604 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.637605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.639010 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:42.639014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.639779 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.639781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:42.639875 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.639877 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:10:42.639942 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:10:42.639982 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":53,"thread_start_us":113,"threads_started":1}
I20260807 15:10:42.640492 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf8e391dacd4bfabfeff85574a8baee" candidate_uuid: "6bf9a1109c884772b4678f3542733def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" is_pre_election: true
I20260807 15:10:42.640650 15920 raft_consensus.cc:2454] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bf9a1109c884772b4678f3542733def in term 0.
I20260807 15:10:42.640910 16213 leader_election.cc:302] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f7dc2a0bac4fcc88c6cddd29ce03b6, 6bf9a1109c884772b4678f3542733def; no voters:
I20260807 15:10:42.640961 16115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf8e391dacd4bfabfeff85574a8baee" candidate_uuid: "6bf9a1109c884772b4678f3542733def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcb43a4f460473b8120ddc7284cd221" is_pre_election: true
I20260807 15:10:42.641042 16409 raft_consensus.cc:2781] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:42.641104 16115 raft_consensus.cc:2454] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bf9a1109c884772b4678f3542733def in term 0.
I20260807 15:10:42.641119 16409 raft_consensus.cc:482] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:42.641149 16409 raft_consensus.cc:3037] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:42.641885 16409 raft_consensus.cc:504] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.642077 16409 leader_election.cc:288] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [CANDIDATE]: Term 1 election: Requested vote from peers 68f7dc2a0bac4fcc88c6cddd29ce03b6 (127.18.129.1:38527), 3fcb43a4f460473b8120ddc7284cd221 (127.18.129.2:45599)
I20260807 15:10:42.642165 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf8e391dacd4bfabfeff85574a8baee" candidate_uuid: "6bf9a1109c884772b4678f3542733def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6"
I20260807 15:10:42.642251 15920 raft_consensus.cc:3037] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:42.642921 16115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf8e391dacd4bfabfeff85574a8baee" candidate_uuid: "6bf9a1109c884772b4678f3542733def" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcb43a4f460473b8120ddc7284cd221"
I20260807 15:10:42.642985 15920 raft_consensus.cc:2454] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bf9a1109c884772b4678f3542733def in term 1.
I20260807 15:10:42.643000 16115 raft_consensus.cc:3037] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:42.643123 16213 leader_election.cc:302] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68f7dc2a0bac4fcc88c6cddd29ce03b6, 6bf9a1109c884772b4678f3542733def; no voters:
I20260807 15:10:42.643240 16409 raft_consensus.cc:2781] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:42.643412 16409 raft_consensus.cc:686] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 1 LEADER]: Becoming Leader. State: Replica: 6bf9a1109c884772b4678f3542733def, State: Running, Role: LEADER
I20260807 15:10:42.643514 16409 consensus_queue.cc:237] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } }
I20260807 15:10:42.643826 16115 raft_consensus.cc:2454] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bf9a1109c884772b4678f3542733def in term 1.
I20260807 15:10:42.644445 15709 catalog_manager.cc:5495] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bf9a1109c884772b4678f3542733def (127.18.129.3). New cstate: current_term: 1 leader_uuid: "6bf9a1109c884772b4678f3542733def" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bf9a1109c884772b4678f3542733def" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 43165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 } health_report { overall_health: UNKNOWN } } }
I20260807 15:10:42.652755 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.650510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40080 (local address 127.18.129.3:43165)
0807 15:10:42.650569 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.650570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.650579 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:10:42.650646 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.650647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.650665 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.650721 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:42.650874 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.651333 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.652076 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.652259 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.652312 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.652453 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:42.652549 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:42.652616 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.652643 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.652662 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:10:42.654379 15920 raft_consensus.cc:1261] T faf8e391dacd4bfabfeff85574a8baee P 68f7dc2a0bac4fcc88c6cddd29ce03b6 [term 1 FOLLOWER]: Refusing update from remote peer 6bf9a1109c884772b4678f3542733def: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:42.654381 16115 raft_consensus.cc:1261] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 1 FOLLOWER]: Refusing update from remote peer 6bf9a1109c884772b4678f3542733def: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:42.654699 16409 consensus_queue.cc:1035] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f7dc2a0bac4fcc88c6cddd29ce03b6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:42.654819 16414 consensus_queue.cc:1035] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcb43a4f460473b8120ddc7284cd221" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 45599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:42.657119 16418 mvcc.cc:205] Tried to move back new op lower bound from 7315928853109948416 to 7315928853068627968. Current Snapshot: MvccSnapshot[applied={T|T < 7315928853109948416}]
I20260807 15:10:42.658140 16420 mvcc.cc:205] Tried to move back new op lower bound from 7315928853109948416 to 7315928853068627968. Current Snapshot: MvccSnapshot[applied={T|T < 7315928853109948416 or (T in {7315928853109948416})}]
I20260807 15:10:42.659499 15709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:42.659582 15709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:42.660266 15709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64c8c532e834452cb0a29494e7b832cd: Sending DeleteTablet for 3 replicas of tablet faf8e391dacd4bfabfeff85574a8baee
I20260807 15:10:42.660589 16279 tablet_service.cc:1514] Processing DeleteTablet for tablet faf8e391dacd4bfabfeff85574a8baee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:42 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:40064
I20260807 15:10:42.660590 16086 tablet_service.cc:1514] Processing DeleteTablet for tablet faf8e391dacd4bfabfeff85574a8baee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:42 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:48126
I20260807 15:10:42.660597 15890 tablet_service.cc:1514] Processing DeleteTablet for tablet faf8e391dacd4bfabfeff85574a8baee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:42 UTC) from {username='kudu', principal='kudu/127.18.129.62@KRBTEST.COM'} at 127.0.0.1:46840
I20260807 15:10:42.660822 16431 tablet_replica.cc:330] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: stopping tablet replica
I20260807 15:10:42.660933 16431 raft_consensus.cc:2229] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:42.661046 16431 raft_consensus.cc:2258] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:42.661053 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 15796
I20260807 15:10:42.670377 16429 tablet_replica.cc:330] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: stopping tablet replica
I20260807 15:10:42.670553 16429 raft_consensus.cc:2229] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:42.670670 16429 raft_consensus.cc:2258] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:42.671988 16429 ts_tablet_manager.cc:1905] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:42.673905 16429 ts_tablet_manager.cc:1918] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:42.674037 16429 log.cc:1196] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/faf8e391dacd4bfabfeff85574a8baee
I20260807 15:10:42.674374 16429 ts_tablet_manager.cc:1939] T faf8e391dacd4bfabfeff85574a8baee P 3fcb43a4f460473b8120ddc7284cd221: Deleting consensus metadata
I20260807 15:10:42.680969 15690 catalog_manager.cc:4841] TS 3fcb43a4f460473b8120ddc7284cd221 (127.18.129.2:45599): tablet faf8e391dacd4bfabfeff85574a8baee (table test-table [id=9f259625517348ebb1635ce506428fd8]) successfully deleted
I20260807 15:10:42.681357 16431 ts_tablet_manager.cc:1905] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:42.683091 16431 ts_tablet_manager.cc:1918] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:42.683180 16431 log.cc:1196] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/faf8e391dacd4bfabfeff85574a8baee
I20260807 15:10:42.683439 16431 ts_tablet_manager.cc:1939] T faf8e391dacd4bfabfeff85574a8baee P 6bf9a1109c884772b4678f3542733def: Deleting consensus metadata
I20260807 15:10:42.683878 15688 catalog_manager.cc:4841] TS 6bf9a1109c884772b4678f3542733def (127.18.129.3:43165): tablet faf8e391dacd4bfabfeff85574a8baee (table test-table [id=9f259625517348ebb1635ce506428fd8]) successfully deleted
W20260807 15:10:42.688177 15687 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:38527 (error 108)
I20260807 15:10:42.688519 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 15991
I20260807 15:10:42.693096 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.688692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:38527 (local address 127.0.0.1:46852)
0807 15:10:42.692984 (+ 4292us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:42.693056 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:38527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4223}
W20260807 15:10:42.693296 15687 catalog_manager.cc:4584] TS 68f7dc2a0bac4fcc88c6cddd29ce03b6 (127.18.129.1:38527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet faf8e391dacd4bfabfeff85574a8baee: Network error: Client connection negotiation failed: client connection to 127.18.129.1:38527: connect: Connection refused (error 111)
I20260807 15:10:42.701202 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 16189
I20260807 15:10:42.711685 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 15660
2026-08-07T15:10:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2002 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T15:10:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:42Z Disabled control of system clock
I20260807 15:10:42.745019 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:36515
--webserver_interface=127.18.129.62
--webserver_port=0
--builtin_ntp_servers=127.18.129.20:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:36515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:42.826498 16439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:42.826738 16439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:42.826757 16439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:42.827935 16439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:42.827983 16439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:42.827996 16439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:42.828007 16439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:42.828018 16439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:42.829334 16439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:36515
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:36515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:42.829537 16439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:42.829779 16439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:42.834875 16444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:42.834904 16445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.834933 16439 server_base.cc:1034] running on GCE node
W20260807 15:10:42.834875 16452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.835302 16439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:42.835554 16439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:42.836688 16439 hybrid_clock.cc:648] HybridClock initialized: now 1786115442836676 us; error 28 us; skew 500 ppm
I20260807 15:10:42.838073 16439 webserver.cc:460] Webserver started at http://127.18.129.62:33185/ using document root <none> and password file <none>
I20260807 15:10:42.838311 16439 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:42.838359 16439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:42.838480 16439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:42.839597 16439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "cd263f87d3c795db1f33f22ed4c17693"
server_key_iv: "637160f07c98ccc4414b9c4390a2d7d6"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.839943 16439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "cd263f87d3c795db1f33f22ed4c17693"
server_key_iv: "637160f07c98ccc4414b9c4390a2d7d6"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.841187 16439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:42.841854 16459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.841988 16439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:42.842080 16439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "cd263f87d3c795db1f33f22ed4c17693"
server_key_iv: "637160f07c98ccc4414b9c4390a2d7d6"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.842149 16439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:42.853783 16439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:42.854106 16439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:42.854241 16439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:42.858250 16439 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:36515
I20260807 15:10:42.858294 16530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:36515 every 8 connection(s)
I20260807 15:10:42.858671 16439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:42.859261 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 16439
I20260807 15:10:42.859360 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:42.859557 18948 external_mini_cluster.cc:1428] Setting key e70c15adf9edbff13519d804feeb5cb9
I20260807 15:10:42.859791 16534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:42.862888 16534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58: Bootstrap starting.
I20260807 15:10:42.863547 16534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:42.863826 16534 log.cc:824] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:42.864549 16534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58: No bootstrap required, opened a new log
I20260807 15:10:42.865146 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:42.860107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.18.129.62:36515)
0807 15:10:42.860296 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:42.860303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:42.860329 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:10:42.860867 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:42.860874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:42.860937 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:42.861056 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:42.861292 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.862192 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.863281 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:42.863734 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:42.863825 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:42.864528 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:42.864532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:42.864532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:42.864707 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:42.864749 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:42.864761 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:42.864817 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:42.864888 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:10:42.864965 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260807 15:10:42.866596 16534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36515 } }
I20260807 15:10:42.866734 16534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:42.866766 16534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5ee8fa11f42a6b8a58e4a15d6dc58, State: Initialized, Role: FOLLOWER
I20260807 15:10:42.866909 16534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36515 } }
I20260807 15:10:42.866983 16534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:42.867023 16534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:42.867071 16534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:42.867624 16534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36515 } }
I20260807 15:10:42.867723 16534 leader_election.cc:302] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04a5ee8fa11f42a6b8a58e4a15d6dc58; no voters:
I20260807 15:10:42.867890 16534 leader_election.cc:288] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:42.867924 16547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:42.868059 16547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [term 1 LEADER]: Becoming Leader. State: Replica: 04a5ee8fa11f42a6b8a58e4a15d6dc58, State: Running, Role: LEADER
I20260807 15:10:42.868150 16534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:42.868194 16547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36515 } }
I20260807 15:10:42.868613 16548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36515 } } }
I20260807 15:10:42.868639 16550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04a5ee8fa11f42a6b8a58e4a15d6dc58. Latest consensus state: current_term: 1 leader_uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04a5ee8fa11f42a6b8a58e4a15d6dc58" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36515 } } }
I20260807 15:10:42.868801 16550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:42.869001 16548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:42.869179 16560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:42.869634 16560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:42.870993 16560 catalog_manager.cc:1347] Generated new cluster ID: f054f948d5b54dd8814142ce92b4f500
I20260807 15:10:42.871048 16560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:42.879737 16560 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:42.880323 16560 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:42.893745 16560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58: Generated new TSK 0
I20260807 15:10:42.893936 16560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:42.895423 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.1:0
--local_ip_for_outbound_sockets=127.18.129.1
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:36515
--builtin_ntp_servers=127.18.129.20:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:42.978135 16573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:42.978348 16573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:42.978379 16573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:42.979557 16573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:42.979598 16573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:42.979631 16573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.1
I20260807 15:10:42.981010 16573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.129.1
--webserver_port=0
--tserver_master_addrs=127.18.129.62:36515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.1
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:42.981243 16573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:42.981536 16573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:42.986545 16583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:42.986553 16581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:42.986594 16580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:42.986832 16573 server_base.cc:1034] running on GCE node
I20260807 15:10:42.987076 16573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:42.987308 16573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:42.988448 16573 hybrid_clock.cc:648] HybridClock initialized: now 1786115442988433 us; error 31 us; skew 500 ppm
I20260807 15:10:42.989646 16573 webserver.cc:460] Webserver started at http://127.18.129.1:35471/ using document root <none> and password file <none>
I20260807 15:10:42.989884 16573 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:42.989934 16573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:42.990058 16573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:42.991181 16573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/data/instance:
uuid: "d5a5067687364b97adb34a7f2c4620a6"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "1ecca81375004d7b477ce314f8e851dd"
server_key_iv: "bafb0db2fd19b7c14c5bde2590c68bf9"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.991499 16573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance:
uuid: "d5a5067687364b97adb34a7f2c4620a6"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "1ecca81375004d7b477ce314f8e851dd"
server_key_iv: "bafb0db2fd19b7c14c5bde2590c68bf9"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.992683 16573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:10:42.993327 16591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:42.993472 16573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:10:42.993561 16573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/wal
uuid: "d5a5067687364b97adb34a7f2c4620a6"
format_stamp: "Formatted at 2026-08-07 15:10:42 on dist-test-slave-5bn2"
server_key: "1ecca81375004d7b477ce314f8e851dd"
server_key_iv: "bafb0db2fd19b7c14c5bde2590c68bf9"
server_key_version: "encryptionkey@0"
I20260807 15:10:42.993618 16573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:43.017053 16573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:43.017381 16573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:43.017530 16573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:43.017764 16573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:43.018112 16573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:43.018148 16573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:43.018179 16573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:43.018203 16573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:43.023787 16573 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.1:45125
I20260807 15:10:43.023850 16752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.1:45125 every 8 connection(s)
I20260807 15:10:43.024163 16573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:43.029210 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.024892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.1:46925 (local address 127.18.129.62:36515)
0807 15:10:43.024955 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.024957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.025189 (+ 232us) server_negotiation.cc:407] Connection header received
0807 15:10:43.025704 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.025706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.025717 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.025767 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:43.026452 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.026983 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.027955 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.028149 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.028205 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.028944 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:43.028946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:43.028946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:43.028951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:43.028967 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:43.028971 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:43.029015 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.029099 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.029129 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:10:43.029403 16757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.024858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:36515 (local address 127.18.129.1:46925)
0807 15:10:43.025067 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.025104 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:43.025607 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:43.025848 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:43.025851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:43.025859 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:43.026363 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.026368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.027073 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.027074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.027868 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.027871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.028224 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.028225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.028343 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.028814 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:43.028815 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:43.028844 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:43.028866 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:43.029032 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:43.029037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:43.029040 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:43.029095 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 15:10:43.029197 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260807 15:10:43.029896 16754 heartbeater.cc:347] Connected to a master server at 127.18.129.62:36515
I20260807 15:10:43.029991 16754 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:43.030154 16754 heartbeater.cc:511] Master 127.18.129.62:36515 requested a full tablet report, sending...
I20260807 15:10:43.030652 16485 ts_manager.cc:194] Registered new tserver with Master: d5a5067687364b97adb34a7f2c4620a6 (127.18.129.1:45125)
I20260807 15:10:43.031186 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 16573
I20260807 15:10:43.031303 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:43.031297 16485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.1:46925
I20260807 15:10:43.031507 18948 external_mini_cluster.cc:1428] Setting key 34e682395f2a67516d56c93ed2c27bf7
I20260807 15:10:43.032401 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.2:0
--local_ip_for_outbound_sockets=127.18.129.2
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:36515
--builtin_ntp_servers=127.18.129.20:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:43.113701 16758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:43.113945 16758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:43.113983 16758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:43.115422 16758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:43.115484 16758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:43.115520 16758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.2
I20260807 15:10:43.117378 16758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.129.2
--webserver_port=0
--tserver_master_addrs=127.18.129.62:36515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.2
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:43.117658 16758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:43.117951 16758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:43.123010 16767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:43.123031 16769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:43.123226 16758 server_base.cc:1034] running on GCE node
W20260807 15:10:43.123243 16773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:43.123464 16758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:43.123696 16758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:43.124847 16758 hybrid_clock.cc:648] HybridClock initialized: now 1786115443124831 us; error 27 us; skew 500 ppm
I20260807 15:10:43.126120 16758 webserver.cc:460] Webserver started at http://127.18.129.2:43147/ using document root <none> and password file <none>
I20260807 15:10:43.126358 16758 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:43.126411 16758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:43.126541 16758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:43.127671 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/data/instance:
uuid: "219bebccd5fd4f83b26f9df0fa467415"
format_stamp: "Formatted at 2026-08-07 15:10:43 on dist-test-slave-5bn2"
server_key: "e191f18c649087ea0980d39c09d135bd"
server_key_iv: "8588272fb9bd8f6920767983eba37065"
server_key_version: "encryptionkey@0"
I20260807 15:10:43.128000 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance:
uuid: "219bebccd5fd4f83b26f9df0fa467415"
format_stamp: "Formatted at 2026-08-07 15:10:43 on dist-test-slave-5bn2"
server_key: "e191f18c649087ea0980d39c09d135bd"
server_key_iv: "8588272fb9bd8f6920767983eba37065"
server_key_version: "encryptionkey@0"
I20260807 15:10:43.129201 16758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:10:43.129848 16780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:43.129993 16758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:10:43.130079 16758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/wal
uuid: "219bebccd5fd4f83b26f9df0fa467415"
format_stamp: "Formatted at 2026-08-07 15:10:43 on dist-test-slave-5bn2"
server_key: "e191f18c649087ea0980d39c09d135bd"
server_key_iv: "8588272fb9bd8f6920767983eba37065"
server_key_version: "encryptionkey@0"
I20260807 15:10:43.130160 16758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:43.155351 16758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:43.155699 16758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:43.155834 16758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:43.156046 16758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:43.156384 16758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:43.156421 16758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:43.156451 16758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:43.156477 16758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:43.161952 16758 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.2:38011
I20260807 15:10:43.162041 16938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.2:38011 every 8 connection(s)
I20260807 15:10:43.162364 16758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:43.167091 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.163080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.2:37125 (local address 127.18.129.62:36515)
0807 15:10:43.163160 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.163164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.163369 (+ 205us) server_negotiation.cc:407] Connection header received
0807 15:10:43.163861 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.163862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.163873 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.163923 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:43.164553 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.165031 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.165936 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.166107 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.166149 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.166920 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:43.166920 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:43.166921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:43.166924 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:43.166933 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:43.166936 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:43.166966 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.167019 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.167034 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:10:43.167282 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.163065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:36515 (local address 127.18.129.2:37125)
0807 15:10:43.163247 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.163284 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:43.163780 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:43.163984 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:43.163987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:43.163995 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:43.164494 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.164498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.165092 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.165092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.165875 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.165877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.166162 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.166162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.166268 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.166789 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:43.166789 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:43.166818 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:43.166842 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:43.166969 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:43.166971 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:43.166974 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:43.167017 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:10:43.167096 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260807 15:10:43.167747 16939 heartbeater.cc:347] Connected to a master server at 127.18.129.62:36515
I20260807 15:10:43.167779 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 16758
I20260807 15:10:43.167852 16939 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:43.167881 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:43.168046 16939 heartbeater.cc:511] Master 127.18.129.62:36515 requested a full tablet report, sending...
I20260807 15:10:43.168118 18948 external_mini_cluster.cc:1428] Setting key cbbbdba64ebaadc023aaf9b623fb1f97
I20260807 15:10:43.168426 16485 ts_manager.cc:194] Registered new tserver with Master: 219bebccd5fd4f83b26f9df0fa467415 (127.18.129.2:38011)
I20260807 15:10:43.168987 16485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.2:37125
I20260807 15:10:43.169282 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.129.3:0
--local_ip_for_outbound_sockets=127.18.129.3
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:36515
--builtin_ntp_servers=127.18.129.20:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:10:43.250339 16943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:43.250554 16943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:43.250576 16943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:43.251720 16943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:43.251760 16943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:43.251785 16943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.129.3
I20260807 15:10:43.253105 16943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.129.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.129.3
--webserver_port=0
--tserver_master_addrs=127.18.129.62:36515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.129.3
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:43.253327 16943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:43.253616 16943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:43.258824 16950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:43.258816 16948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:43.258916 16943 server_base.cc:1034] running on GCE node
W20260807 15:10:43.258816 16954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:43.259287 16943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:43.259541 16943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:43.260694 16943 hybrid_clock.cc:648] HybridClock initialized: now 1786115443260678 us; error 28 us; skew 500 ppm
I20260807 15:10:43.261960 16943 webserver.cc:460] Webserver started at http://127.18.129.3:39883/ using document root <none> and password file <none>
I20260807 15:10:43.262184 16943 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:43.262236 16943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:43.262364 16943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:43.263479 16943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/data/instance:
uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c"
format_stamp: "Formatted at 2026-08-07 15:10:43 on dist-test-slave-5bn2"
server_key: "a7e0e2525771960b694efef65ae23482"
server_key_iv: "67e0cb398be88526b3112d64db52821f"
server_key_version: "encryptionkey@0"
I20260807 15:10:43.263828 16943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance:
uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c"
format_stamp: "Formatted at 2026-08-07 15:10:43 on dist-test-slave-5bn2"
server_key: "a7e0e2525771960b694efef65ae23482"
server_key_iv: "67e0cb398be88526b3112d64db52821f"
server_key_version: "encryptionkey@0"
I20260807 15:10:43.265038 16943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:43.265722 16961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:43.265875 16943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:43.265962 16943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/wal
uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c"
format_stamp: "Formatted at 2026-08-07 15:10:43 on dist-test-slave-5bn2"
server_key: "a7e0e2525771960b694efef65ae23482"
server_key_iv: "67e0cb398be88526b3112d64db52821f"
server_key_version: "encryptionkey@0"
I20260807 15:10:43.266021 16943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:43.276204 16943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:43.276463 16943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:43.276590 16943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:43.276788 16943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:43.277105 16943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:43.277140 16943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:43.277168 16943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:43.277194 16943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:43.282987 16943 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.3:34753
I20260807 15:10:43.283043 17124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.3:34753 every 8 connection(s)
I20260807 15:10:43.283353 16943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:43.284758 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 16943
I20260807 15:10:43.284883 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:43.285089 18948 external_mini_cluster.cc:1428] Setting key 8dcac8787d5bbc214364d4dc70c81ea8
I20260807 15:10:43.288578 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.284137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:52541 (local address 127.18.129.62:36515)
0807 15:10:43.284217 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.284220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.284453 (+ 233us) server_negotiation.cc:407] Connection header received
0807 15:10:43.285022 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.285024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.285037 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.285089 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:43.285814 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.286313 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.287281 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.287477 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.287531 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.288310 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:43.288311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:43.288312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:43.288316 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:43.288331 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:43.288335 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:43.288381 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.288468 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.288494 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:10:43.288779 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.284108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:36515 (local address 127.18.129.3:52541)
0807 15:10:43.284290 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.284324 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:43.284906 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:43.285160 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:43.285164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:43.285175 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:43.285729 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.285734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.286402 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.286404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.287199 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.287201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.287544 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.287545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.287665 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.288173 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:43.288173 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:43.288202 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:43.288226 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:43.288395 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:43.288401 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:43.288404 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:43.288463 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:10:43.288565 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260807 15:10:43.289249 17125 heartbeater.cc:347] Connected to a master server at 127.18.129.62:36515
I20260807 15:10:43.289340 17125 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:43.289489 17125 heartbeater.cc:511] Master 127.18.129.62:36515 requested a full tablet report, sending...
I20260807 15:10:43.289835 16485 ts_manager.cc:194] Registered new tserver with Master: a5e5c927176a4ca6b2da7fa5a1bb637c (127.18.129.3:34753)
I20260807 15:10:43.290180 16485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.129.3:52541
I20260807 15:10:43.296861 18948 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:10:43.301028 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.298248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.18.129.62:36515)
0807 15:10:43.298307 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.298309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.298395 (+ 86us) server_negotiation.cc:407] Connection header received
0807 15:10:43.298523 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.298524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.298535 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.298583 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:43.298736 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.299242 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.299970 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.300154 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.300206 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.300808 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:43.300810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:43.300810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:43.300815 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:43.300829 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:43.300833 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:43.300873 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.300929 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.300950 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:10:43.302681 16485 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:43.304251 16485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:43.310267 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.306451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.18.129.2:38011)
0807 15:10:43.306635 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.306639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.306648 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:10:43.306685 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.306687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.306702 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.306793 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:43.306988 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.308204 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.309398 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.309885 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.309949 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.310049 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.310144 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.310182 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":76,"threads_started":1}
I20260807 15:10:43.310480 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.306205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:45125 (local address 127.0.0.1:49232)
0807 15:10:43.306361 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.306400 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:43.306464 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:43.306875 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:43.306876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:43.306878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:43.306917 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.306917 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.307990 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.307991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.309167 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.309170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.310074 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.310075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.310159 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.310164 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:10:43.310195 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 15:10:43.310235 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":6,"thread_start_us":48,"threads_started":1}
I20260807 15:10:43.310487 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.306210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.3:34753 (local address 127.0.0.1:39860)
0807 15:10:43.306473 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.306482 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:43.306520 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:43.306990 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:43.306991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:43.306992 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:43.307026 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.307027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.308321 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.308322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.309162 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.309165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.310174 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.310175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.310226 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.310229 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:43.310271 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:10:43.310303 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260807 15:10:43.310544 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.306293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49232 (local address 127.18.129.1:45125)
0807 15:10:43.306706 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.306710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.306718 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:10:43.306749 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.306750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.306763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.306989 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:43.307054 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.307909 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.309242 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.309974 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.310302 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.310407 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.310432 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.310472 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":95,"threads_started":1}
I20260807 15:10:43.310263 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.306214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:38011 (local address 127.0.0.1:40760)
0807 15:10:43.306578 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.306585 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:43.306617 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:43.306813 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:43.306816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:43.306818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:43.306912 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.306914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.308289 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.308290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.309330 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.309332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.309960 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.309961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.310036 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.310050 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:10:43.310130 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:10:43.310182 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":26,"thread_start_us":108,"threads_started":1}
I20260807 15:10:43.311072 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.306417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.18.129.3:34753)
0807 15:10:43.306626 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.306630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.306641 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:10:43.306685 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.306687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.306702 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.306793 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:43.307151 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.308253 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.309247 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.309972 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.310041 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.310140 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.310526 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.310991 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
I20260807 15:10:43.311336 16658 tablet_service.cc:1467] Processing CreateTablet for tablet 50c8fb7974214e528280e51b8f571cc7 (DEFAULT_TABLE table=test-table [id=cc69bc86c2154d03b83c8c827f5adf0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:43.311466 16845 tablet_service.cc:1467] Processing CreateTablet for tablet 50c8fb7974214e528280e51b8f571cc7 (DEFAULT_TABLE table=test-table [id=cc69bc86c2154d03b83c8c827f5adf0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:43.311751 16658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c8fb7974214e528280e51b8f571cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:43.311815 16845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c8fb7974214e528280e51b8f571cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:43.311731 17025 tablet_service.cc:1467] Processing CreateTablet for tablet 50c8fb7974214e528280e51b8f571cc7 (DEFAULT_TABLE table=test-table [id=cc69bc86c2154d03b83c8c827f5adf0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:43.312029 17025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c8fb7974214e528280e51b8f571cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:43.314291 17145 tablet_bootstrap.cc:492] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6: Bootstrap starting.
I20260807 15:10:43.314934 17145 tablet_bootstrap.cc:654] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:43.315187 17146 tablet_bootstrap.cc:492] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: Bootstrap starting.
I20260807 15:10:43.315248 17147 tablet_bootstrap.cc:492] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: Bootstrap starting.
I20260807 15:10:43.315268 17145 log.cc:824] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:43.315761 17146 tablet_bootstrap.cc:654] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:43.315853 17147 tablet_bootstrap.cc:654] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:43.316075 17146 log.cc:824] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:43.316114 17145 tablet_bootstrap.cc:492] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6: No bootstrap required, opened a new log
I20260807 15:10:43.316133 17147 log.cc:824] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:43.316187 17145 ts_tablet_manager.cc:1397] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:10:43.316783 17147 tablet_bootstrap.cc:492] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: No bootstrap required, opened a new log
I20260807 15:10:43.316843 17147 ts_tablet_manager.cc:1397] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:43.317042 17146 tablet_bootstrap.cc:492] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: No bootstrap required, opened a new log
I20260807 15:10:43.317121 17146 ts_tablet_manager.cc:1397] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:43.318609 17145 raft_consensus.cc:348] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.318789 17145 raft_consensus.cc:374] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:43.318826 17145 raft_consensus.cc:729] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a5067687364b97adb34a7f2c4620a6, State: Initialized, Role: FOLLOWER
I20260807 15:10:43.318755 17147 raft_consensus.cc:348] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.318923 17147 raft_consensus.cc:374] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:43.318965 17147 raft_consensus.cc:729] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 219bebccd5fd4f83b26f9df0fa467415, State: Initialized, Role: FOLLOWER
I20260807 15:10:43.318991 17145 consensus_queue.cc:260] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.319087 17147 consensus_queue.cc:260] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.319271 17145 ts_tablet_manager.cc:1428] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 15:10:43.319329 17147 ts_tablet_manager.cc:1428] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:10:43.319398 16754 heartbeater.cc:503] Master 127.18.129.62:36515 was elected leader, sending a full tablet report...
I20260807 15:10:43.319417 16939 heartbeater.cc:503] Master 127.18.129.62:36515 was elected leader, sending a full tablet report...
I20260807 15:10:43.320163 17146 raft_consensus.cc:348] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.320292 17146 raft_consensus.cc:374] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:43.320336 17146 raft_consensus.cc:729] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e5c927176a4ca6b2da7fa5a1bb637c, State: Initialized, Role: FOLLOWER
I20260807 15:10:43.320459 17146 consensus_queue.cc:260] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.320712 17146 ts_tablet_manager.cc:1428] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:10:43.320766 17125 heartbeater.cc:503] Master 127.18.129.62:36515 was elected leader, sending a full tablet report...
I20260807 15:10:43.387984 17153 raft_consensus.cc:482] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:43.388118 17153 raft_consensus.cc:504] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.388494 17153 leader_election.cc:288] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a5067687364b97adb34a7f2c4620a6 (127.18.129.1:45125), 219bebccd5fd4f83b26f9df0fa467415 (127.18.129.2:38011)
I20260807 15:10:43.391342 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.388725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:32843 (local address 127.18.129.1:45125)
0807 15:10:43.388781 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.388783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.388795 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:10:43.388826 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.388827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.388841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.388896 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:43.389069 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.389596 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.390641 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.391018 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.391084 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.391093 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.391223 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.391262 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:10:43.391361 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.388798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.129.3:44697 (local address 127.18.129.2:38011)
0807 15:10:43.388842 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.388844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.388954 (+ 110us) server_negotiation.cc:407] Connection header received
0807 15:10:43.388988 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.388989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.388999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.389054 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:43.389209 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.389678 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.390629 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.391018 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.391080 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.391089 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.391258 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.391299 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:10:43.391361 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.388661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:45125 (local address 127.18.129.3:32843)
0807 15:10:43.388720 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.388731 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:43.388774 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:43.388906 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:43.388907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:43.388909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:43.388982 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.388984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.389667 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.389668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.390536 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.390538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.391091 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.391092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.391159 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.391163 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:43.391203 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:10:43.391245 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":3}
I20260807 15:10:43.391371 17154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.388688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.2:38011 (local address 127.18.129.3:44697)
0807 15:10:43.388883 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.388892 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:43.388941 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:43.389062 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:43.389062 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:43.389063 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:43.389150 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.389151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.389750 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.389751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.390543 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:43.390546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.391096 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.391097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:43.391183 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.391186 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:10:43.391234 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:10:43.391270 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":122,"threads_started":1}
I20260807 15:10:43.391718 16689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c8fb7974214e528280e51b8f571cc7" candidate_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a5067687364b97adb34a7f2c4620a6" is_pre_election: true
I20260807 15:10:43.391898 16689 raft_consensus.cc:2454] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5e5c927176a4ca6b2da7fa5a1bb637c in term 0.
I20260807 15:10:43.391942 16873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c8fb7974214e528280e51b8f571cc7" candidate_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "219bebccd5fd4f83b26f9df0fa467415" is_pre_election: true
I20260807 15:10:43.392086 16873 raft_consensus.cc:2454] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5e5c927176a4ca6b2da7fa5a1bb637c in term 0.
I20260807 15:10:43.392131 16966 leader_election.cc:302] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5e5c927176a4ca6b2da7fa5a1bb637c, d5a5067687364b97adb34a7f2c4620a6; no voters:
I20260807 15:10:43.392333 17153 raft_consensus.cc:2781] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:43.392378 17153 raft_consensus.cc:482] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:43.392393 17153 raft_consensus.cc:3037] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:43.393067 17153 raft_consensus.cc:504] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.393245 17153 leader_election.cc:288] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [CANDIDATE]: Term 1 election: Requested vote from peers d5a5067687364b97adb34a7f2c4620a6 (127.18.129.1:45125), 219bebccd5fd4f83b26f9df0fa467415 (127.18.129.2:38011)
I20260807 15:10:43.393342 16873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c8fb7974214e528280e51b8f571cc7" candidate_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "219bebccd5fd4f83b26f9df0fa467415"
I20260807 15:10:43.393397 16873 raft_consensus.cc:3037] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:43.394052 16873 raft_consensus.cc:2454] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5e5c927176a4ca6b2da7fa5a1bb637c in term 1.
I20260807 15:10:43.394157 16689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c8fb7974214e528280e51b8f571cc7" candidate_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a5067687364b97adb34a7f2c4620a6"
I20260807 15:10:43.394225 16689 raft_consensus.cc:3037] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:43.394347 16962 leader_election.cc:302] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 219bebccd5fd4f83b26f9df0fa467415, a5e5c927176a4ca6b2da7fa5a1bb637c; no voters:
I20260807 15:10:43.394457 17153 raft_consensus.cc:2781] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:43.394591 17153 raft_consensus.cc:686] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 1 LEADER]: Becoming Leader. State: Replica: a5e5c927176a4ca6b2da7fa5a1bb637c, State: Running, Role: LEADER
I20260807 15:10:43.394670 17153 consensus_queue.cc:237] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } }
I20260807 15:10:43.395121 16689 raft_consensus.cc:2454] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5e5c927176a4ca6b2da7fa5a1bb637c in term 1.
I20260807 15:10:43.395540 16483 catalog_manager.cc:5495] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c reported cstate change: term changed from 0 to 1, leader changed from <none> to a5e5c927176a4ca6b2da7fa5a1bb637c (127.18.129.3). New cstate: current_term: 1 leader_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5e5c927176a4ca6b2da7fa5a1bb637c" member_type: VOTER last_known_addr { host: "127.18.129.3" port: 34753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 } health_report { overall_health: UNKNOWN } } }
I20260807 15:10:43.400560 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.398372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.18.129.3:34753)
0807 15:10:43.398431 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:43.398432 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:43.398440 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:10:43.398469 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:43.398469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:43.398486 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:43.398547 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:43.398667 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.399159 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.399863 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:43.400066 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:43.400118 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:43.400256 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:43.400367 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:43.400437 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:43.400454 (+ 17us) server_negotiation.cc:299] Negotiation successful
0807 15:10:43.400470 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:10:43.402204 16873 raft_consensus.cc:1261] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 1 FOLLOWER]: Refusing update from remote peer a5e5c927176a4ca6b2da7fa5a1bb637c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:43.402222 16689 raft_consensus.cc:1261] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [term 1 FOLLOWER]: Refusing update from remote peer a5e5c927176a4ca6b2da7fa5a1bb637c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:43.402509 17153 consensus_queue.cc:1035] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "219bebccd5fd4f83b26f9df0fa467415" member_type: VOTER last_known_addr { host: "127.18.129.2" port: 38011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:43.402652 17155 consensus_queue.cc:1035] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a5067687364b97adb34a7f2c4620a6" member_type: VOTER last_known_addr { host: "127.18.129.1" port: 45125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:43.405063 17160 mvcc.cc:205] Tried to move back new op lower bound from 7315928856173268992 to 7315928856145309696. Current Snapshot: MvccSnapshot[applied={T|T < 7315928856173268992}]
I20260807 15:10:43.405941 17162 mvcc.cc:205] Tried to move back new op lower bound from 7315928856173268992 to 7315928856145309696. Current Snapshot: MvccSnapshot[applied={T|T < 7315928856173268992}]
I20260807 15:10:43.407461 16483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:43.407553 16483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:43.408301 16483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04a5ee8fa11f42a6b8a58e4a15d6dc58: Sending DeleteTablet for 3 replicas of tablet 50c8fb7974214e528280e51b8f571cc7
I20260807 15:10:43.408663 16658 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c8fb7974214e528280e51b8f571cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:43 UTC) from {username='slave'} at 127.0.0.1:49232
I20260807 15:10:43.408708 16845 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c8fb7974214e528280e51b8f571cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:43 UTC) from {username='slave'} at 127.0.0.1:40760
I20260807 15:10:43.408882 17025 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c8fb7974214e528280e51b8f571cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:43 UTC) from {username='slave'} at 127.0.0.1:39860
I20260807 15:10:43.408938 17177 tablet_replica.cc:330] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: stopping tablet replica
I20260807 15:10:43.409039 17177 raft_consensus.cc:2229] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:43.409152 17177 raft_consensus.cc:2258] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:43.409155 17178 tablet_replica.cc:330] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: stopping tablet replica
I20260807 15:10:43.409296 17178 raft_consensus.cc:2229] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:43.409404 17176 tablet_replica.cc:330] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6: stopping tablet replica
I20260807 15:10:43.409467 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 16573
I20260807 15:10:43.409518 17176 raft_consensus.cc:2229] T 50c8fb7974214e528280e51b8f571cc7 P d5a5067687364b97adb34a7f2c4620a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:43.421994 17178 raft_consensus.cc:2258] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:43.422295 17177 ts_tablet_manager.cc:1905] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:43.422693 17178 ts_tablet_manager.cc:1905] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:43.424199 17177 ts_tablet_manager.cc:1918] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:43.424293 17177 log.cc:1196] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-1/wal/wals/50c8fb7974214e528280e51b8f571cc7
I20260807 15:10:43.424546 17178 ts_tablet_manager.cc:1918] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:43.424608 17177 ts_tablet_manager.cc:1939] T 50c8fb7974214e528280e51b8f571cc7 P 219bebccd5fd4f83b26f9df0fa467415: Deleting consensus metadata
I20260807 15:10:43.424736 17178 log.cc:1196] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: Deleting WAL directory at /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115197351496-18948-0/minicluster-data/ts-2/wal/wals/50c8fb7974214e528280e51b8f571cc7
I20260807 15:10:43.425083 17178 ts_tablet_manager.cc:1939] T 50c8fb7974214e528280e51b8f571cc7 P a5e5c927176a4ca6b2da7fa5a1bb637c: Deleting consensus metadata
I20260807 15:10:43.425105 16460 catalog_manager.cc:4841] TS 219bebccd5fd4f83b26f9df0fa467415 (127.18.129.2:38011): tablet 50c8fb7974214e528280e51b8f571cc7 (table test-table [id=cc69bc86c2154d03b83c8c827f5adf0c]) successfully deleted
I20260807 15:10:43.425617 16465 catalog_manager.cc:4841] TS a5e5c927176a4ca6b2da7fa5a1bb637c (127.18.129.3:34753): tablet 50c8fb7974214e528280e51b8f571cc7 (table test-table [id=cc69bc86c2154d03b83c8c827f5adf0c]) successfully deleted
W20260807 15:10:43.426739 16464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.129.1:45125 (error 108)
I20260807 15:10:43.427297 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:43.427151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.1:45125 (local address 127.0.0.1:49238)
0807 15:10:43.427209 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:43.427257 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.1:45125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:10:43.427441 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 16758
W20260807 15:10:43.436908 16464 catalog_manager.cc:4584] TS d5a5067687364b97adb34a7f2c4620a6 (127.18.129.1:45125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50c8fb7974214e528280e51b8f571cc7: Network error: Client connection negotiation failed: client connection to 127.18.129.1:45125: connect: Connection refused (error 111)
I20260807 15:10:43.441279 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 16943
I20260807 15:10:43.453613 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 16439
2026-08-07T15:10:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (744 ms)
[----------] 6 tests from EncryptionPolicyTest (8757 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:43 dist-test-slave-5bn2 krb5kdc[17184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:43 dist-test-slave-5bn2 krb5kdc[17184](info): set up 2 sockets
Aug 07 15:10:43 dist-test-slave-5bn2 krb5kdc[17184](info): commencing operation
krb5kdc: starting...
W20260807 15:10:44.602245 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:44 dist-test-slave-5bn2 krb5kdc[17184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:44.718935 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:44.799124 17200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:44.799327 17200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:44.799350 17200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:44.800437 17200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:44.800472 17200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:44.800484 17200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:44.800495 17200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:44.800508 17200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:44.801913 17200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37693
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:44.802101 17200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:44.802263 17200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:44.807065 17211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:44.807093 17200 server_base.cc:1034] running on GCE node
W20260807 15:10:44.807060 17208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:44.807096 17207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:44.807495 17200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:44.807777 17200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:44.808925 17200 hybrid_clock.cc:648] HybridClock initialized: now 1786115444808910 us; error 28 us; skew 500 ppm
Aug 07 15:10:44 dist-test-slave-5bn2 krb5kdc[17184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115444, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:44.811295 17200 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:44.811741 17200 webserver.cc:460] Webserver started at http://127.0.0.1:36209/ using document root <none> and password file <none>
I20260807 15:10:44.811935 17200 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:44.811982 17200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:44.812098 17200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:44.813108 17200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "ec4f67e99c4b45a497697ade8a04f0af"
format_stamp: "Formatted at 2026-08-07 15:10:44 on dist-test-slave-5bn2"
server_key: "cee3b8ded06ee0f0e81612944c3909fa"
server_key_iv: "4fb6cd64089d484b6d927cac995e93c5"
server_key_version: "encryptionkey@0"
I20260807 15:10:44.813438 17200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "ec4f67e99c4b45a497697ade8a04f0af"
format_stamp: "Formatted at 2026-08-07 15:10:44 on dist-test-slave-5bn2"
server_key: "cee3b8ded06ee0f0e81612944c3909fa"
server_key_iv: "4fb6cd64089d484b6d927cac995e93c5"
server_key_version: "encryptionkey@0"
I20260807 15:10:44.814675 17200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:44.815352 17219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:44.815501 17200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:44.815593 17200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "ec4f67e99c4b45a497697ade8a04f0af"
format_stamp: "Formatted at 2026-08-07 15:10:44 on dist-test-slave-5bn2"
server_key: "cee3b8ded06ee0f0e81612944c3909fa"
server_key_iv: "4fb6cd64089d484b6d927cac995e93c5"
server_key_version: "encryptionkey@0"
I20260807 15:10:44.815654 17200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:44.831866 17200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:44.833642 17200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:44.833775 17200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:44.837790 17200 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37693
I20260807 15:10:44.837826 17289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37693 every 8 connection(s)
I20260807 15:10:44.838160 17200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:44.839022 17290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:44.841274 17290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af: Bootstrap starting.
I20260807 15:10:44.841837 17290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:44.842099 17290 log.cc:824] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:44.842753 17290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af: No bootstrap required, opened a new log
I20260807 15:10:44.843371 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 17200
I20260807 15:10:44.843482 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:44.843667 18948 external_mini_cluster.cc:1428] Setting key e4c992f4fa44cadac23c38be661323d0
I20260807 15:10:44.844797 17290 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f67e99c4b45a497697ade8a04f0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37693 } }
I20260807 15:10:44.844969 17290 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:44.845005 17290 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec4f67e99c4b45a497697ade8a04f0af, State: Initialized, Role: FOLLOWER
I20260807 15:10:44.845162 17290 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f67e99c4b45a497697ade8a04f0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37693 } }
I20260807 15:10:44.845229 17290 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:44.845250 17290 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:44.845283 17290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:44.846318 17290 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f67e99c4b45a497697ade8a04f0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37693 } }
I20260807 15:10:44.846426 17290 leader_election.cc:302] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec4f67e99c4b45a497697ade8a04f0af; no voters:
I20260807 15:10:44.846593 17290 leader_election.cc:288] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:44.846639 17296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:44.846781 17296 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [term 1 LEADER]: Becoming Leader. State: Replica: ec4f67e99c4b45a497697ade8a04f0af, State: Running, Role: LEADER
I20260807 15:10:44.846819 17290 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:44.846900 17296 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f67e99c4b45a497697ade8a04f0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37693 } }
I20260807 15:10:44.847231 17296 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec4f67e99c4b45a497697ade8a04f0af. Latest consensus state: current_term: 1 leader_uuid: "ec4f67e99c4b45a497697ade8a04f0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f67e99c4b45a497697ade8a04f0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37693 } } }
I20260807 15:10:44.847304 17296 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:44.847545 17304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:44.848084 17304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:44.848380 17297 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec4f67e99c4b45a497697ade8a04f0af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec4f67e99c4b45a497697ade8a04f0af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37693 } } }
I20260807 15:10:44.848464 17297 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:44.849606 17304 catalog_manager.cc:1347] Generated new cluster ID: 847df2f6fb7449ca837873421f9d2462
I20260807 15:10:44.849658 17304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:10:44 dist-test-slave-5bn2 krb5kdc[17184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:10:44.854326 17295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:44.844194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60703 (local address 127.0.0.1:37693)
0807 15:10:44.844377 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:44.844386 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:44.844422 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:10:44.844768 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:44.844777 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:44.844840 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:44.844986 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:44.845698 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:44.847132 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:44.848176 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:44.848778 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:44.848878 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:44.851268 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:44.851272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:44.851272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:44.851284 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:44.852812 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:44.853186 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:44.853189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:44.853190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:44.853301 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:44.853512 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:44.853512 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:44.853513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:44.853781 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:44.853953 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:44.854064 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 15:10:44.854170 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260807 15:10:44.855262 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:44.862114 17295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:44.856510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50711 (local address 127.0.0.1:37693)
0807 15:10:44.856547 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:44.856549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:44.856675 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:10:44.856726 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:44.856727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:44.856736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:44.856781 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:44.857251 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:44.857730 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:44.858391 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:44.858621 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:44.858685 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:44.859799 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:44.859800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:44.859801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:44.859808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:44.861061 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:44.861401 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:44.861402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:44.861402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:44.861452 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:44.861650 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:44.861651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:44.861651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:44.861750 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:44.861852 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:44.861959 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 15:10:44.862016 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:10:44.871308 17304 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:44.871896 17304 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:44.881289 17304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec4f67e99c4b45a497697ade8a04f0af: Generated new TSK 0
I20260807 15:10:44.881474 17304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:44.891080 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 17200
W20260807 15:10:44.900156 17182 connection.cc:535] client connection to 127.0.0.1:37693 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37693): Connection reset by peer (error 104)
2026-08-07T15:10:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:44 dist-test-slave-5bn2 krb5kdc[17334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:44 dist-test-slave-5bn2 krb5kdc[17334](info): set up 2 sockets
Aug 07 15:10:44 dist-test-slave-5bn2 krb5kdc[17334](info): commencing operation
krb5kdc: starting...
W20260807 15:10:46.024791 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:46 dist-test-slave-5bn2 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:46.145663 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:46.226181 17352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:46.226387 17352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:46.226418 17352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:46.227556 17352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:46.227604 17352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:46.227623 17352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:46.227639 17352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:46.227654 17352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:46.229113 17352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40567
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:46.229352 17352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:46.229548 17352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:46.234594 17358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:46.234638 17363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:46.234638 17357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:46.234946 17352 server_base.cc:1034] running on GCE node
I20260807 15:10:46.235221 17352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:46.235543 17352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:46.236683 17352 hybrid_clock.cc:648] HybridClock initialized: now 1786115446236675 us; error 28 us; skew 500 ppm
Aug 07 15:10:46 dist-test-slave-5bn2 krb5kdc[17334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115446, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:46.244426 17352 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:46.244853 17352 webserver.cc:460] Webserver started at http://127.0.0.1:46609/ using document root <none> and password file <none>
I20260807 15:10:46.245049 17352 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:46.245098 17352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:46.245219 17352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:46.246263 17352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "c067d3eea5014dddb869884af2d4f409"
format_stamp: "Formatted at 2026-08-07 15:10:46 on dist-test-slave-5bn2"
server_key: "d467c08dfed5581dac72c729de911883"
server_key_iv: "a4d9e4083726133248ce3f8bc164e768"
server_key_version: "encryptionkey@0"
I20260807 15:10:46.246622 17352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "c067d3eea5014dddb869884af2d4f409"
format_stamp: "Formatted at 2026-08-07 15:10:46 on dist-test-slave-5bn2"
server_key: "d467c08dfed5581dac72c729de911883"
server_key_iv: "a4d9e4083726133248ce3f8bc164e768"
server_key_version: "encryptionkey@0"
I20260807 15:10:46.247929 17352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:46.248646 17370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:46.248799 17352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:46.248888 17352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "c067d3eea5014dddb869884af2d4f409"
format_stamp: "Formatted at 2026-08-07 15:10:46 on dist-test-slave-5bn2"
server_key: "d467c08dfed5581dac72c729de911883"
server_key_iv: "a4d9e4083726133248ce3f8bc164e768"
server_key_version: "encryptionkey@0"
I20260807 15:10:46.248955 17352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:46.262352 17352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:46.264287 17352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:46.264406 17352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:46.268244 17352 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40567
I20260807 15:10:46.268292 17440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40567 every 8 connection(s)
I20260807 15:10:46.268596 17352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:46.269464 17442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:46.270548 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 17352
I20260807 15:10:46.270679 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:46.270929 18948 external_mini_cluster.cc:1428] Setting key fe4deaa7d4ff72378658ed03f4bb32a9
I20260807 15:10:46.272317 17442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409: Bootstrap starting.
I20260807 15:10:46.272946 17442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:46.273204 17442 log.cc:824] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:46.273828 17442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409: No bootstrap required, opened a new log
I20260807 15:10:46.275718 17442 raft_consensus.cc:348] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c067d3eea5014dddb869884af2d4f409" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40567 } }
I20260807 15:10:46.275853 17442 raft_consensus.cc:374] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:46.275887 17442 raft_consensus.cc:729] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c067d3eea5014dddb869884af2d4f409, State: Initialized, Role: FOLLOWER
I20260807 15:10:46.276013 17442 consensus_queue.cc:260] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c067d3eea5014dddb869884af2d4f409" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40567 } }
I20260807 15:10:46.276094 17442 raft_consensus.cc:388] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:46.276144 17442 raft_consensus.cc:482] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:46.276196 17442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:46.276252 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:46.271436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52327 (local address 127.0.0.1:40567)
0807 15:10:46.271604 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:46.271614 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:46.271643 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:10:46.272142 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:46.272149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:46.272194 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:46.272302 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:46.272562 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:46.273530 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:46.274315 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:46.274728 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:46.274822 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:46.275554 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:46.275558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:46.275558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:46.275707 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:46.275737 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:46.275748 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:46.275814 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:46.276005 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 15:10:46.276037 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260807 15:10:46.276799 17442 raft_consensus.cc:504] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c067d3eea5014dddb869884af2d4f409" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40567 } }
I20260807 15:10:46.276901 17442 leader_election.cc:302] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c067d3eea5014dddb869884af2d4f409; no voters:
I20260807 15:10:46.277055 17442 leader_election.cc:288] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:46.277098 17448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:46.277251 17448 raft_consensus.cc:686] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [term 1 LEADER]: Becoming Leader. State: Replica: c067d3eea5014dddb869884af2d4f409, State: Running, Role: LEADER
I20260807 15:10:46.277302 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:46.277298 17442 sys_catalog.cc:564] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:46.277333 17448 consensus_queue.cc:237] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c067d3eea5014dddb869884af2d4f409" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40567 } }
I20260807 15:10:46.277659 17448 sys_catalog.cc:455] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c067d3eea5014dddb869884af2d4f409. Latest consensus state: current_term: 1 leader_uuid: "c067d3eea5014dddb869884af2d4f409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c067d3eea5014dddb869884af2d4f409" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40567 } } }
I20260807 15:10:46.277724 17448 sys_catalog.cc:458] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:46.277858 17449 sys_catalog.cc:455] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c067d3eea5014dddb869884af2d4f409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c067d3eea5014dddb869884af2d4f409" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40567 } } }
I20260807 15:10:46.277906 17449 sys_catalog.cc:458] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:46.278008 17454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:46.278631 17454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:46.280169 17454 catalog_manager.cc:1347] Generated new cluster ID: 7489c2a9c36e48b19e89cade036f38bc
I20260807 15:10:46.280218 17454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:46.281919 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:46.279084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41923 (local address 127.0.0.1:40567)
0807 15:10:46.279119 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:46.279120 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:46.279358 (+ 238us) server_negotiation.cc:407] Connection header received
0807 15:10:46.279387 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:46.279388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:46.279398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:46.279443 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:46.279557 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:46.280122 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:46.280849 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:46.281068 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:46.281122 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:46.281690 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:46.281692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:46.281692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:46.281696 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:46.281710 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:46.281714 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:46.281764 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:46.281831 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 15:10:46.281840 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:10:46.295455 17454 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:46.296170 17454 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:46.303943 17454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c067d3eea5014dddb869884af2d4f409: Generated new TSK 0
I20260807 15:10:46.304127 17454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:46.313186 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 17352
W20260807 15:10:46.327765 17332 connection.cc:535] client connection to 127.0.0.1:40567 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40567): Connection reset by peer (error 104)
2026-08-07T15:10:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:46 dist-test-slave-5bn2 krb5kdc[17484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:46 dist-test-slave-5bn2 krb5kdc[17484](info): set up 2 sockets
Aug 07 15:10:46 dist-test-slave-5bn2 krb5kdc[17484](info): commencing operation
krb5kdc: starting...
W20260807 15:10:47.437896 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:47 dist-test-slave-5bn2 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:47.558000 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:47.640489 17502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:47.640694 17502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:47.640714 17502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:47.641886 17502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:47.641932 17502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:47.641945 17502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:47.641956 17502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:47.641979 17502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:47.643496 17502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:47.643733 17502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:47.643944 17502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:47.648886 17508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:47.648892 17507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:47.649099 17502 server_base.cc:1034] running on GCE node
W20260807 15:10:47.649013 17512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:47.649430 17502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:47.649714 17502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:47.650888 17502 hybrid_clock.cc:648] HybridClock initialized: now 1786115447650870 us; error 27 us; skew 500 ppm
Aug 07 15:10:47 dist-test-slave-5bn2 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115447, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:47.653329 17502 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:47.653753 17502 webserver.cc:460] Webserver started at http://127.0.0.1:44645/ using document root <none> and password file <none>
I20260807 15:10:47.653978 17502 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:47.654029 17502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:47.654155 17502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:47.655233 17502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "1b71688d255646308a15267222075134"
format_stamp: "Formatted at 2026-08-07 15:10:47 on dist-test-slave-5bn2"
server_key: "b085b751b30958ff93df444e389149e8"
server_key_iv: "154d4fe3555d9369a1bf9a1cd8335a4d"
server_key_version: "encryptionkey@0"
I20260807 15:10:47.655556 17502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "1b71688d255646308a15267222075134"
format_stamp: "Formatted at 2026-08-07 15:10:47 on dist-test-slave-5bn2"
server_key: "b085b751b30958ff93df444e389149e8"
server_key_iv: "154d4fe3555d9369a1bf9a1cd8335a4d"
server_key_version: "encryptionkey@0"
I20260807 15:10:47.656775 17502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:47.657477 17524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:47.657660 17502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:47.657748 17502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "1b71688d255646308a15267222075134"
format_stamp: "Formatted at 2026-08-07 15:10:47 on dist-test-slave-5bn2"
server_key: "b085b751b30958ff93df444e389149e8"
server_key_iv: "154d4fe3555d9369a1bf9a1cd8335a4d"
server_key_version: "encryptionkey@0"
I20260807 15:10:47.657811 17502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:47.661075 17502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:47.663236 17502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:47.663369 17502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:47.667554 17502 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33029
I20260807 15:10:47.667587 17599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33029 every 8 connection(s)
I20260807 15:10:47.667915 17502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:47.668977 17600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:47.671804 17600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134: Bootstrap starting.
I20260807 15:10:47.672399 17600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:47.672685 17600 log.cc:824] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:47.672963 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 17502
I20260807 15:10:47.673086 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:47.673319 18948 external_mini_cluster.cc:1428] Setting key 9aaf9d7b992372d5b9f56e6412bb63c2
I20260807 15:10:47.673499 17600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134: No bootstrap required, opened a new log
I20260807 15:10:47.675797 17600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b71688d255646308a15267222075134" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260807 15:10:47.675979 17600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:47.676019 17600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b71688d255646308a15267222075134, State: Initialized, Role: FOLLOWER
I20260807 15:10:47.676160 17600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b71688d255646308a15267222075134" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260807 15:10:47.676213 17606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:47.673864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41319 (local address 127.0.0.1:33029)
0807 15:10:47.674057 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:47.674065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:47.674091 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:10:47.674815 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:47.674822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:47.674878 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:47.675025 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:47.675648 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:47.675651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:47.675651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:47.675655 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:47.675694 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:47.675704 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:47.675755 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:47.675835 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 15:10:47.675869 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260807 15:10:47.676260 17600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:47.676296 17600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:47.676337 17600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:47.676941 17600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b71688d255646308a15267222075134" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260807 15:10:47.677042 17600 leader_election.cc:302] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b71688d255646308a15267222075134; no voters:
I20260807 15:10:47.677099 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:47.677212 17600 leader_election.cc:288] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:47.677244 17607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:47.677405 17607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [term 1 LEADER]: Becoming Leader. State: Replica: 1b71688d255646308a15267222075134, State: Running, Role: LEADER
I20260807 15:10:47.677428 17600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:47.677492 17607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b71688d255646308a15267222075134" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260807 15:10:47.677848 17607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b71688d255646308a15267222075134. Latest consensus state: current_term: 1 leader_uuid: "1b71688d255646308a15267222075134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b71688d255646308a15267222075134" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } } }
I20260807 15:10:47.677927 17607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:47.678077 17608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b71688d255646308a15267222075134" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b71688d255646308a15267222075134" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } } }
I20260807 15:10:47.678149 17608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:47.678260 17617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:47.678767 17617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:47.680370 17617 catalog_manager.cc:1347] Generated new cluster ID: 5794595df2da4448993ebb80086b0305
I20260807 15:10:47.680425 17617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:47.682950 17606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:47.679112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45403 (local address 127.0.0.1:33029)
0807 15:10:47.679150 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:47.679151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:47.679403 (+ 252us) server_negotiation.cc:407] Connection header received
0807 15:10:47.679494 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:47.679495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:47.679500 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:47.679567 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:47.680200 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:47.680201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:47.680202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:47.680206 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:47.680220 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:47.680224 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:47.680251 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:47.680348 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 15:10:47.680360 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:10:47.687474 17617 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:47.687613 17617 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:47.696897 17617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b71688d255646308a15267222075134: Generated new TSK 0
I20260807 15:10:47.697104 17617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:47.699954 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 17502
2026-08-07T15:10:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:47 dist-test-slave-5bn2 krb5kdc[17646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:47 dist-test-slave-5bn2 krb5kdc[17646](info): set up 2 sockets
Aug 07 15:10:47 dist-test-slave-5bn2 krb5kdc[17646](info): commencing operation
krb5kdc: starting...
W20260807 15:10:48.835587 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:48 dist-test-slave-5bn2 krb5kdc[17646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:48.953455 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:49.033425 17661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:49.033650 17661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:49.033682 17661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:49.034832 17661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:49.034899 17661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:49.034919 17661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:49.034935 17661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:49.034950 17661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:49.036454 17661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:49.036715 17661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:49.036913 17661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:49.041709 17668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:49.041702 17669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:49.041697 17674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:49.042038 17661 server_base.cc:1034] running on GCE node
I20260807 15:10:49.042295 17661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:49.042588 17661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:49.043733 17661 hybrid_clock.cc:648] HybridClock initialized: now 1786115449043717 us; error 33 us; skew 500 ppm
Aug 07 15:10:49 dist-test-slave-5bn2 krb5kdc[17646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115449, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:49.046234 17661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:49.046638 17661 webserver.cc:460] Webserver started at http://127.0.0.1:35015/ using document root <none> and password file <none>
I20260807 15:10:49.046820 17661 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:49.046892 17661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:49.047019 17661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:49.048062 17661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "62f7815eddb6467cb901e92ecb6a40e0"
format_stamp: "Formatted at 2026-08-07 15:10:49 on dist-test-slave-5bn2"
server_key: "df87dcd3a97753bfb65b403064767f95"
server_key_iv: "a1f2728792ad48d3ec435f5250f26e78"
server_key_version: "encryptionkey@0"
I20260807 15:10:49.048398 17661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "62f7815eddb6467cb901e92ecb6a40e0"
format_stamp: "Formatted at 2026-08-07 15:10:49 on dist-test-slave-5bn2"
server_key: "df87dcd3a97753bfb65b403064767f95"
server_key_iv: "a1f2728792ad48d3ec435f5250f26e78"
server_key_version: "encryptionkey@0"
I20260807 15:10:49.049652 17661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:10:49.050335 17682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:49.050523 17661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:10:49.050602 17661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "62f7815eddb6467cb901e92ecb6a40e0"
format_stamp: "Formatted at 2026-08-07 15:10:49 on dist-test-slave-5bn2"
server_key: "df87dcd3a97753bfb65b403064767f95"
server_key_iv: "a1f2728792ad48d3ec435f5250f26e78"
server_key_version: "encryptionkey@0"
I20260807 15:10:49.050669 17661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:49.078116 17661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:49.080380 17661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:49.080523 17661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:49.084462 17661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46005
I20260807 15:10:49.084523 17754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46005 every 8 connection(s)
I20260807 15:10:49.084841 17661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:49.085690 17755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:49.088001 17755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0: Bootstrap starting.
I20260807 15:10:49.088323 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 17661
I20260807 15:10:49.088443 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:49.088644 18948 external_mini_cluster.cc:1428] Setting key f5adf6f9835d79959c716a1a4e5c55bf
I20260807 15:10:49.088730 17755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:49.089017 17755 log.cc:824] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:49.089771 17755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0: No bootstrap required, opened a new log
I20260807 15:10:49.092036 17755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f7815eddb6467cb901e92ecb6a40e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } }
I20260807 15:10:49.092216 17755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:49.092253 17755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62f7815eddb6467cb901e92ecb6a40e0, State: Initialized, Role: FOLLOWER
I20260807 15:10:49.092381 17755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f7815eddb6467cb901e92ecb6a40e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } }
I20260807 15:10:49.092446 17755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:49.092474 17755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:49.092514 17755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:49.093056 17755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f7815eddb6467cb901e92ecb6a40e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } }
I20260807 15:10:49.093169 17755 leader_election.cc:302] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62f7815eddb6467cb901e92ecb6a40e0; no voters:
I20260807 15:10:49.093326 17755 leader_election.cc:288] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:49.093384 17761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:49.093549 17755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:49.093554 17761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [term 1 LEADER]: Becoming Leader. State: Replica: 62f7815eddb6467cb901e92ecb6a40e0, State: Running, Role: LEADER
I20260807 15:10:49.093700 17761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f7815eddb6467cb901e92ecb6a40e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } }
I20260807 15:10:49.094275 17764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62f7815eddb6467cb901e92ecb6a40e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f7815eddb6467cb901e92ecb6a40e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } } }
I20260807 15:10:49.094275 17763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62f7815eddb6467cb901e92ecb6a40e0. Latest consensus state: current_term: 1 leader_uuid: "62f7815eddb6467cb901e92ecb6a40e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62f7815eddb6467cb901e92ecb6a40e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } } }
I20260807 15:10:49.094733 17763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:49.094446 17764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:49.095561 17772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:49.096063 17772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:10:49 dist-test-slave-5bn2 krb5kdc[17646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:10:49.097620 17772 catalog_manager.cc:1347] Generated new cluster ID: 5a5dc4efcfe24be89cdbb755d32252d1
I20260807 15:10:49.097673 17772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:49.100909 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:49.089154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58867 (local address 127.0.0.1:46005)
0807 15:10:49.089304 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:49.089313 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:49.089346 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:10:49.089905 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:49.089915 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:49.089975 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:49.090134 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:49.090789 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:49.092202 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:49.093046 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:49.093975 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:49.094404 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:49.097168 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:49.097172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:49.097173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:49.097183 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:49.099236 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:49.099628 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:49.099632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:49.099632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:49.099752 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:49.099938 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:49.099939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:49.099939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:49.100258 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:49.100429 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:49.100515 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:10:49.100668 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260807 15:10:49.101864 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:49.110337 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:49.103477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44721 (local address 127.0.0.1:46005)
0807 15:10:49.103518 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:49.103520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:49.103531 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:10:49.103580 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:49.103581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:49.103591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:49.103657 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:49.104255 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:49.104954 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:49.105897 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:49.106212 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:49.106291 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:49.107569 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:49.107570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:49.107570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:49.107578 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:49.109142 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:49.109482 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:49.109483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:49.109483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:49.109547 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:49.109719 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:49.109720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:49.109720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:49.109859 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:49.109962 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:49.110046 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:10:49.110141 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:10:49.117774 17772 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:49.118299 17772 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:49.123221 17772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62f7815eddb6467cb901e92ecb6a40e0: Generated new TSK 0
I20260807 15:10:49.123389 17772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:49.126333 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 17661
2026-08-07T15:10:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:49 dist-test-slave-5bn2 krb5kdc[17799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:49 dist-test-slave-5bn2 krb5kdc[17799](info): set up 2 sockets
Aug 07 15:10:49 dist-test-slave-5bn2 krb5kdc[17799](info): commencing operation
krb5kdc: starting...
W20260807 15:10:50.271234 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:50 dist-test-slave-5bn2 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:50.393790 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43731
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:50.475612 17814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:50.475834 17814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:50.475868 17814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:50.477026 17814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:50.477075 17814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:50.477093 17814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:50.477108 17814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:50.477123 17814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:50.478633 17814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:50.478904 17814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:50.479120 17814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:50.483950 17824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:50.484176 17814 server_base.cc:1034] running on GCE node
W20260807 15:10:50.483950 17821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:50.483982 17822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:50.484512 17814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:50.484797 17814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:50.485944 17814 hybrid_clock.cc:648] HybridClock initialized: now 1786115450485926 us; error 32 us; skew 500 ppm
Aug 07 15:10:50 dist-test-slave-5bn2 krb5kdc[17799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115450, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:50.488376 17814 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:50.488839 17814 webserver.cc:460] Webserver started at http://127.0.0.1:35381/ using document root <none> and password file <none>
I20260807 15:10:50.489068 17814 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:50.489120 17814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:50.489248 17814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:50.490307 17814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb"
format_stamp: "Formatted at 2026-08-07 15:10:50 on dist-test-slave-5bn2"
server_key: "bf6e5ef34ce8c3034a2aace4b08ff9dc"
server_key_iv: "0d69903e34ee8bbeefee1609fd3aa3ca"
server_key_version: "encryptionkey@0"
I20260807 15:10:50.490648 17814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb"
format_stamp: "Formatted at 2026-08-07 15:10:50 on dist-test-slave-5bn2"
server_key: "bf6e5ef34ce8c3034a2aace4b08ff9dc"
server_key_iv: "0d69903e34ee8bbeefee1609fd3aa3ca"
server_key_version: "encryptionkey@0"
I20260807 15:10:50.491938 17814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 15:10:50.492597 17832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:50.492743 17814 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:50.492830 17814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb"
format_stamp: "Formatted at 2026-08-07 15:10:50 on dist-test-slave-5bn2"
server_key: "bf6e5ef34ce8c3034a2aace4b08ff9dc"
server_key_iv: "0d69903e34ee8bbeefee1609fd3aa3ca"
server_key_version: "encryptionkey@0"
I20260807 15:10:50.492898 17814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:50.513437 17814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:50.515693 17814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:50.515851 17814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:50.519927 17814 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43731
I20260807 15:10:50.519954 17906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43731 every 8 connection(s)
I20260807 15:10:50.520315 17814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:50.521198 17908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:50.523592 17908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb: Bootstrap starting.
I20260807 15:10:50.524168 17908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:50.524434 17908 log.cc:824] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:50.525120 17908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb: No bootstrap required, opened a new log
I20260807 15:10:50.526728 17908 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43731 } }
I20260807 15:10:50.526907 17908 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:50.526947 17908 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac39c915fb0c4b69a653f4e9ab54f2fb, State: Initialized, Role: FOLLOWER
I20260807 15:10:50.527089 17908 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43731 } }
I20260807 15:10:50.527150 17908 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:50.527182 17908 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:50.527228 17908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:50.527748 17908 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43731 } }
I20260807 15:10:50.527839 17908 leader_election.cc:302] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac39c915fb0c4b69a653f4e9ab54f2fb; no voters:
I20260807 15:10:50.528013 17908 leader_election.cc:288] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:50.528103 17912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:50.528254 17912 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [term 1 LEADER]: Becoming Leader. State: Replica: ac39c915fb0c4b69a653f4e9ab54f2fb, State: Running, Role: LEADER
I20260807 15:10:50.528258 17908 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:50.528369 17912 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43731 } }
I20260807 15:10:50.528721 17913 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43731 } } }
I20260807 15:10:50.528805 17913 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:50.528929 17914 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac39c915fb0c4b69a653f4e9ab54f2fb. Latest consensus state: current_term: 1 leader_uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac39c915fb0c4b69a653f4e9ab54f2fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43731 } } }
I20260807 15:10:50.529042 17914 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:50.529042 17922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:50.529197 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 17814
I20260807 15:10:50.529330 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:50.529546 18948 external_mini_cluster.cc:1428] Setting key 954474d966c2e929600086ce9aa5d3f6
I20260807 15:10:50.529647 17922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:50.531621 17922 catalog_manager.cc:1347] Generated new cluster ID: a43a03f7df4144f1b847f5fce21a7e64
I20260807 15:10:50.531679 17922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:50.535225 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:50.530189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58777 (local address 127.0.0.1:43731)
0807 15:10:50.530354 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:50.530363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:50.530778 (+ 415us) server_negotiation.cc:407] Connection header received
0807 15:10:50.531122 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:50.531131 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:50.531192 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:50.531325 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:50.531675 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:50.532626 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:50.533396 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:50.533824 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:50.533931 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:50.534575 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:50.534579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:50.534580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:50.534747 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:50.534779 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:50.534790 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:50.534867 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:50.534946 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 15:10:50.535009 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260807 15:10:50.536059 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:50.539041 17922 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:50.539641 17922 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:50.540359 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:50.537335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33525 (local address 127.0.0.1:43731)
0807 15:10:50.537370 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:50.537372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:50.537465 (+ 93us) server_negotiation.cc:407] Connection header received
0807 15:10:50.537519 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:50.537520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:50.537538 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:50.537582 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:50.537704 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:50.538153 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:50.539014 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:50.539320 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:50.539400 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:50.540113 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:50.540115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:50.540115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:50.540120 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:50.540137 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:50.540142 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:50.540199 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:50.540239 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 15:10:50.540281 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260807 15:10:50.543852 17922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac39c915fb0c4b69a653f4e9ab54f2fb: Generated new TSK 0
I20260807 15:10:50.544029 17922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:50.547197 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 17814
2026-08-07T15:10:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:50 dist-test-slave-5bn2 krb5kdc[17948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:50 dist-test-slave-5bn2 krb5kdc[17948](info): set up 2 sockets
Aug 07 15:10:50 dist-test-slave-5bn2 krb5kdc[17948](info): commencing operation
krb5kdc: starting...
W20260807 15:10:51.678658 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:51 dist-test-slave-5bn2 krb5kdc[17948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:51.799880 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40317
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:51.884037 17966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:51.884382 17966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:51.884438 17966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:51.886036 17966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:51.886132 17966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:51.886174 17966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:51.886193 17966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:51.886209 17966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:51.887873 17966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:51.888118 17966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:51.888337 17966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:51.893188 17976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:51.893211 17974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:51.893311 17966 server_base.cc:1034] running on GCE node
W20260807 15:10:51.893198 17978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:51.893672 17966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:51.893957 17966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:51.895088 17966 hybrid_clock.cc:648] HybridClock initialized: now 1786115451895072 us; error 31 us; skew 500 ppm
Aug 07 15:10:51 dist-test-slave-5bn2 krb5kdc[17948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:51.900890 17966 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:51.901368 17966 webserver.cc:460] Webserver started at http://127.0.0.1:44239/ using document root <none> and password file <none>
I20260807 15:10:51.901610 17966 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:51.901659 17966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:51.901790 17966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:51.902827 17966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "b89a3d0ba80643a2a3bd91de215475c4"
format_stamp: "Formatted at 2026-08-07 15:10:51 on dist-test-slave-5bn2"
server_key: "cb053752d3b15d89ef9ee37cbcb20de1"
server_key_iv: "fed885a9aa2bdbe13dec0f1e709475d8"
server_key_version: "encryptionkey@0"
I20260807 15:10:51.903204 17966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "b89a3d0ba80643a2a3bd91de215475c4"
format_stamp: "Formatted at 2026-08-07 15:10:51 on dist-test-slave-5bn2"
server_key: "cb053752d3b15d89ef9ee37cbcb20de1"
server_key_iv: "fed885a9aa2bdbe13dec0f1e709475d8"
server_key_version: "encryptionkey@0"
I20260807 15:10:51.904472 17966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:51.905200 17987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:51.905347 17966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:51.905433 17966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "b89a3d0ba80643a2a3bd91de215475c4"
format_stamp: "Formatted at 2026-08-07 15:10:51 on dist-test-slave-5bn2"
server_key: "cb053752d3b15d89ef9ee37cbcb20de1"
server_key_iv: "fed885a9aa2bdbe13dec0f1e709475d8"
server_key_version: "encryptionkey@0"
I20260807 15:10:51.905500 17966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:51.908834 17966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:51.910955 17966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:51.911087 17966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:51.915088 17966 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40317
I20260807 15:10:51.915120 18061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40317 every 8 connection(s)
I20260807 15:10:51.915443 17966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:51.916440 18062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:51.918723 18062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4: Bootstrap starting.
I20260807 15:10:51.919332 18062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:51.919603 18062 log.cc:824] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:51.920262 18062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4: No bootstrap required, opened a new log
I20260807 15:10:51.921854 18062 raft_consensus.cc:348] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89a3d0ba80643a2a3bd91de215475c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40317 } }
I20260807 15:10:51.921998 18062 raft_consensus.cc:374] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:51.922031 18062 raft_consensus.cc:729] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b89a3d0ba80643a2a3bd91de215475c4, State: Initialized, Role: FOLLOWER
I20260807 15:10:51.922179 18062 consensus_queue.cc:260] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89a3d0ba80643a2a3bd91de215475c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40317 } }
I20260807 15:10:51.922242 18062 raft_consensus.cc:388] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:51.922264 18062 raft_consensus.cc:482] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:51.922312 18062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:51.922820 18062 raft_consensus.cc:504] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89a3d0ba80643a2a3bd91de215475c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40317 } }
I20260807 15:10:51.922950 18062 leader_election.cc:302] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b89a3d0ba80643a2a3bd91de215475c4; no voters:
I20260807 15:10:51.923120 18062 leader_election.cc:288] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:51.923152 18065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:51.923278 18065 raft_consensus.cc:686] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [term 1 LEADER]: Becoming Leader. State: Replica: b89a3d0ba80643a2a3bd91de215475c4, State: Running, Role: LEADER
I20260807 15:10:51.923374 18062 sys_catalog.cc:564] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:51.923372 18065 consensus_queue.cc:237] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89a3d0ba80643a2a3bd91de215475c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40317 } }
I20260807 15:10:51.923782 18067 sys_catalog.cc:455] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b89a3d0ba80643a2a3bd91de215475c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89a3d0ba80643a2a3bd91de215475c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40317 } } }
I20260807 15:10:51.923862 18067 sys_catalog.cc:458] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:51.924005 18068 sys_catalog.cc:455] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b89a3d0ba80643a2a3bd91de215475c4. Latest consensus state: current_term: 1 leader_uuid: "b89a3d0ba80643a2a3bd91de215475c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89a3d0ba80643a2a3bd91de215475c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40317 } } }
I20260807 15:10:51.924095 18068 sys_catalog.cc:458] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:51.924285 18074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:51.924811 18074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:51.925395 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 17966
I20260807 15:10:51.925516 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:51.925704 18948 external_mini_cluster.cc:1428] Setting key e12f1d78f99b77a3c5b4c956969827cb
I20260807 15:10:51.926304 18074 catalog_manager.cc:1347] Generated new cluster ID: 04c8e6e8df5243a09c81d0b97c762246
I20260807 15:10:51.926354 18074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:51.933530 18074 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:51.933693 18074 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:51.934901 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:51.926172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39137 (local address 127.0.0.1:40317)
0807 15:10:51.933308 (+ 7136us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:51.933318 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:51.933354 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:10:51.933480 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:51.933487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:51.933521 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:51.933662 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:51.934437 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:51.934441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:51.934441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:51.934446 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:51.934490 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:51.934504 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:51.934576 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:51.934629 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 15:10:51.934669 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7089,"thread_start_us":93,"threads_started":1}
I20260807 15:10:51.935797 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:51.938958 18074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b89a3d0ba80643a2a3bd91de215475c4: Generated new TSK 0
I20260807 15:10:51.939133 18074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:51.939983 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:51.937289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59115 (local address 127.0.0.1:40317)
0807 15:10:51.937326 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:51.937328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:51.939048 (+ 1720us) server_negotiation.cc:407] Connection header received
0807 15:10:51.939141 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:51.939142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:51.939147 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:51.939207 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:51.939787 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:51.939789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:51.939789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:51.939794 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:51.939811 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:51.939815 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:51.939874 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:51.939907 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 15:10:51.939920 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:10:51.941980 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 17966
2026-08-07T15:10:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:52 dist-test-slave-5bn2 krb5kdc[18101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:52 dist-test-slave-5bn2 krb5kdc[18101](info): set up 2 sockets
Aug 07 15:10:52 dist-test-slave-5bn2 krb5kdc[18101](info): commencing operation
krb5kdc: starting...
W20260807 15:10:53.077020 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:53 dist-test-slave-5bn2 krb5kdc[18101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:53.198076 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46833
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:53.279682 18119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:53.279904 18119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:53.279927 18119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:53.281000 18119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:53.281034 18119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:53.281047 18119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:53.281059 18119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:53.281068 18119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:53.282531 18119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:53.282755 18119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:53.283006 18119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:53.287983 18127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:53.288156 18129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:53.288197 18126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:53.288298 18119 server_base.cc:1034] running on GCE node
I20260807 15:10:53.288532 18119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:53.288801 18119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:53.289937 18119 hybrid_clock.cc:648] HybridClock initialized: now 1786115453289924 us; error 27 us; skew 500 ppm
Aug 07 15:10:53 dist-test-slave-5bn2 krb5kdc[18101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:53.292426 18119 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:53.292870 18119 webserver.cc:460] Webserver started at http://127.0.0.1:45329/ using document root <none> and password file <none>
I20260807 15:10:53.293066 18119 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:53.293105 18119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:53.293195 18119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:53.294225 18119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "b91b8a6d665a43c6843f61b93fa06fd5"
format_stamp: "Formatted at 2026-08-07 15:10:53 on dist-test-slave-5bn2"
server_key: "c26a0f8076653b7b5d6e79eac68708e3"
server_key_iv: "046922416a2c935c0c85b45734af6c8e"
server_key_version: "encryptionkey@0"
I20260807 15:10:53.294531 18119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "b91b8a6d665a43c6843f61b93fa06fd5"
format_stamp: "Formatted at 2026-08-07 15:10:53 on dist-test-slave-5bn2"
server_key: "c26a0f8076653b7b5d6e79eac68708e3"
server_key_iv: "046922416a2c935c0c85b45734af6c8e"
server_key_version: "encryptionkey@0"
I20260807 15:10:53.295768 18119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:10:53.296408 18139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:53.296563 18119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:53.296662 18119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "b91b8a6d665a43c6843f61b93fa06fd5"
format_stamp: "Formatted at 2026-08-07 15:10:53 on dist-test-slave-5bn2"
server_key: "c26a0f8076653b7b5d6e79eac68708e3"
server_key_iv: "046922416a2c935c0c85b45734af6c8e"
server_key_version: "encryptionkey@0"
I20260807 15:10:53.296720 18119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:53.337491 18119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:53.339684 18119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:53.339823 18119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:53.343721 18119 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46833
I20260807 15:10:53.343762 18221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46833 every 8 connection(s)
I20260807 15:10:53.344112 18119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:53.345043 18223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:53.347373 18223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5: Bootstrap starting.
I20260807 15:10:53.347908 18223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:53.348182 18223 log.cc:824] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:53.348838 18223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5: No bootstrap required, opened a new log
I20260807 15:10:53.350430 18223 raft_consensus.cc:348] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b91b8a6d665a43c6843f61b93fa06fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } }
I20260807 15:10:53.350569 18223 raft_consensus.cc:374] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:53.350591 18223 raft_consensus.cc:729] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b91b8a6d665a43c6843f61b93fa06fd5, State: Initialized, Role: FOLLOWER
I20260807 15:10:53.350695 18223 consensus_queue.cc:260] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b91b8a6d665a43c6843f61b93fa06fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } }
I20260807 15:10:53.350745 18223 raft_consensus.cc:388] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:53.350769 18223 raft_consensus.cc:482] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:53.350798 18223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:53.351300 18223 raft_consensus.cc:504] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b91b8a6d665a43c6843f61b93fa06fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } }
I20260807 15:10:53.351388 18223 leader_election.cc:302] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b91b8a6d665a43c6843f61b93fa06fd5; no voters:
I20260807 15:10:53.351532 18223 leader_election.cc:288] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:53.351584 18227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:53.351755 18227 raft_consensus.cc:686] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [term 1 LEADER]: Becoming Leader. State: Replica: b91b8a6d665a43c6843f61b93fa06fd5, State: Running, Role: LEADER
I20260807 15:10:53.351768 18223 sys_catalog.cc:564] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:53.351866 18227 consensus_queue.cc:237] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b91b8a6d665a43c6843f61b93fa06fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } }
I20260807 15:10:53.352231 18228 sys_catalog.cc:455] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b91b8a6d665a43c6843f61b93fa06fd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b91b8a6d665a43c6843f61b93fa06fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } } }
I20260807 15:10:53.352313 18228 sys_catalog.cc:458] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:53.352356 18229 sys_catalog.cc:455] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b91b8a6d665a43c6843f61b93fa06fd5. Latest consensus state: current_term: 1 leader_uuid: "b91b8a6d665a43c6843f61b93fa06fd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b91b8a6d665a43c6843f61b93fa06fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } } }
I20260807 15:10:53.352420 18229 sys_catalog.cc:458] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:53.353015 18237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:53.353541 18237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:53.353727 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 18119
I20260807 15:10:53.353883 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:53.354096 18948 external_mini_cluster.cc:1428] Setting key e84025aa5c4f1151774453c0ecad22c9
I20260807 15:10:53.355055 18237 catalog_manager.cc:1347] Generated new cluster ID: 9b26739b8f2643db8f962e4b68d8b515
I20260807 15:10:53.355110 18237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:10:53 dist-test-slave-5bn2 krb5kdc[18101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:10:53.365833 18254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:53.354586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:36995 (local address 127.0.0.1:46833)
0807 15:10:53.354746 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:53.354755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:53.355123 (+ 368us) server_negotiation.cc:407] Connection header received
0807 15:10:53.355242 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:53.355249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:53.355305 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:53.355432 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:53.356214 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:53.357610 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:53.358662 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:53.359301 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:53.359391 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:53.362204 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:53.362208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:53.362208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:53.362220 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:53.364176 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:53.364590 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:53.364593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:53.364594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:53.364725 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:53.364916 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:53.364917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:53.364917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:53.365246 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:53.365390 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:53.365491 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 15:10:53.365594 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260807 15:10:53.366973 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:53.375422 18254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:53.368357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:38503 (local address 127.0.0.1:46833)
0807 15:10:53.368390 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:53.368392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:53.368511 (+ 119us) server_negotiation.cc:407] Connection header received
0807 15:10:53.368573 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:53.368574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:53.368586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:53.368631 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:53.369178 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:53.369956 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:53.370931 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:53.371271 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:53.371340 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:53.372663 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:53.372665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:53.372665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:53.372672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:53.374197 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:53.374612 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:53.374613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:53.374614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:53.374674 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:53.374911 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:53.374912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:53.374912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:53.375058 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:53.375164 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:53.375259 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 15:10:53.375323 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:10:53.380538 18237 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:53.381037 18237 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:53.386008 18237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b91b8a6d665a43c6843f61b93fa06fd5: Generated new TSK 0
I20260807 15:10:53.386183 18237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:53.390405 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 18119
W20260807 15:10:53.400228 18096 connection.cc:535] client connection to 127.0.0.1:46833 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46833): Connection reset by peer (error 104)
2026-08-07T15:10:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:53 dist-test-slave-5bn2 krb5kdc[18265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:53 dist-test-slave-5bn2 krb5kdc[18265](info): set up 2 sockets
Aug 07 15:10:53 dist-test-slave-5bn2 krb5kdc[18265](info): commencing operation
krb5kdc: starting...
W20260807 15:10:54.523514 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:54 dist-test-slave-5bn2 krb5kdc[18265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:54.647936 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33473
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:54.731011 18283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:54.731242 18283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:54.731266 18283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:54.732424 18283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:54.732466 18283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:54.732481 18283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:54.732492 18283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:54.732503 18283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:54.734009 18283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:54.734263 18283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:54.734503 18283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:54.739198 18291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:54.739204 18293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:54.739262 18289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:54.739372 18283 server_base.cc:1034] running on GCE node
I20260807 15:10:54.739701 18283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:54.739976 18283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:54.741137 18283 hybrid_clock.cc:648] HybridClock initialized: now 1786115454741106 us; error 25 us; skew 500 ppm
Aug 07 15:10:54 dist-test-slave-5bn2 krb5kdc[18265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:54.744100 18283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:54.744530 18283 webserver.cc:460] Webserver started at http://127.0.0.1:39997/ using document root <none> and password file <none>
I20260807 15:10:54.744740 18283 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:54.744786 18283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:54.744910 18283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:54.745992 18283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "0ee99ed1a6c146d795758750a678702d"
format_stamp: "Formatted at 2026-08-07 15:10:54 on dist-test-slave-5bn2"
server_key: "afd2502c2ef8f0d0e8c82ca05b141f46"
server_key_iv: "9eca173383a8bcb75d1214c705db1000"
server_key_version: "encryptionkey@0"
I20260807 15:10:54.746343 18283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "0ee99ed1a6c146d795758750a678702d"
format_stamp: "Formatted at 2026-08-07 15:10:54 on dist-test-slave-5bn2"
server_key: "afd2502c2ef8f0d0e8c82ca05b141f46"
server_key_iv: "9eca173383a8bcb75d1214c705db1000"
server_key_version: "encryptionkey@0"
I20260807 15:10:54.747733 18283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:54.748533 18302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:54.748706 18283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:54.748792 18283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "0ee99ed1a6c146d795758750a678702d"
format_stamp: "Formatted at 2026-08-07 15:10:54 on dist-test-slave-5bn2"
server_key: "afd2502c2ef8f0d0e8c82ca05b141f46"
server_key_iv: "9eca173383a8bcb75d1214c705db1000"
server_key_version: "encryptionkey@0"
I20260807 15:10:54.748850 18283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:54.762609 18283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:54.764396 18283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:54.764505 18283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:54.768291 18283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33473
I20260807 15:10:54.768332 18374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33473 every 8 connection(s)
I20260807 15:10:54.768646 18283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:54.769508 18376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:54.771924 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d: Bootstrap starting.
I20260807 15:10:54.772492 18376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:54.772774 18376 log.cc:824] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:54.773417 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 18283
I20260807 15:10:54.773517 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d: No bootstrap required, opened a new log
I20260807 15:10:54.773535 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:54.773752 18948 external_mini_cluster.cc:1428] Setting key 85f87a0604d2dafac2e2068a713e356c
I20260807 15:10:54.775456 18376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee99ed1a6c146d795758750a678702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33473 } }
I20260807 15:10:54.775607 18376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:54.775642 18376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee99ed1a6c146d795758750a678702d, State: Initialized, Role: FOLLOWER
I20260807 15:10:54.775791 18376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee99ed1a6c146d795758750a678702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33473 } }
I20260807 15:10:54.775870 18376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:54.775902 18376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:54.775949 18376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:54.776500 18376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee99ed1a6c146d795758750a678702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33473 } }
I20260807 15:10:54.776604 18376 leader_election.cc:302] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ee99ed1a6c146d795758750a678702d; no voters:
I20260807 15:10:54.776772 18376 leader_election.cc:288] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:54.776808 18382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:54.776916 18382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [term 1 LEADER]: Becoming Leader. State: Replica: 0ee99ed1a6c146d795758750a678702d, State: Running, Role: LEADER
I20260807 15:10:54.777040 18376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:54.777057 18382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee99ed1a6c146d795758750a678702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33473 } }
I20260807 15:10:54.777468 18382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ee99ed1a6c146d795758750a678702d. Latest consensus state: current_term: 1 leader_uuid: "0ee99ed1a6c146d795758750a678702d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee99ed1a6c146d795758750a678702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33473 } } }
I20260807 15:10:54.777577 18382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:54.777997 18385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ee99ed1a6c146d795758750a678702d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ee99ed1a6c146d795758750a678702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33473 } } }
I20260807 15:10:54.778069 18385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:54.778071 18393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:54.778646 18393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:54.780395 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:54.774295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:42869 (local address 127.0.0.1:33473)
0807 15:10:54.774481 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:54.774489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:54.774521 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:10:54.775425 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:54.775432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:54.775488 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:54.775615 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:54.775812 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:54.777147 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:54.778216 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:54.778796 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:54.779016 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:54.779718 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:54.779722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:54.779722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:54.779899 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:54.779932 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:54.779944 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:54.780022 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:54.780202 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 15:10:54.780227 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260807 15:10:54.781414 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:54.784108 18393 catalog_manager.cc:1347] Generated new cluster ID: f0bf3cfbf8ea4878af190a6537678fb6
I20260807 15:10:54.784159 18393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:54.786300 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:54.782947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:43217 (local address 127.0.0.1:33473)
0807 15:10:54.783044 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:54.783046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:54.783095 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:10:54.783154 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:54.783154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:54.783164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:54.783208 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:54.783328 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:54.783941 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:54.784898 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:54.785212 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:54.785261 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:54.785970 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:54.785972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:54.785972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:54.785976 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:54.785990 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:54.785993 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:54.786041 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:54.786220 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 15:10:54.786231 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 15:10:54.815253 18393 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:54.815799 18393 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:54.828810 18393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ee99ed1a6c146d795758750a678702d: Generated new TSK 0
I20260807 15:10:54.829042 18393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:54.839823 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 18283
W20260807 15:10:54.851115 18263 connection.cc:535] client connection to 127.0.0.1:33473 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33473): Connection reset by peer (error 104)
2026-08-07T15:10:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:54 dist-test-slave-5bn2 krb5kdc[18416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:54 dist-test-slave-5bn2 krb5kdc[18416](info): set up 2 sockets
Aug 07 15:10:54 dist-test-slave-5bn2 krb5kdc[18416](info): commencing operation
krb5kdc: starting...
W20260807 15:10:55.984563 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:56 dist-test-slave-5bn2 krb5kdc[18416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:56.111853 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39351
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:56.195568 18436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:56.195784 18436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:56.195808 18436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:56.196909 18436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:56.196945 18436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:56.196959 18436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:56.196969 18436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:56.196979 18436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:56.198434 18436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:56.198647 18436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:56.198820 18436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:56.203629 18442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:56.203706 18441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:56.203814 18445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:56.203828 18436 server_base.cc:1034] running on GCE node
I20260807 15:10:56.204159 18436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:56.204428 18436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:56.205570 18436 hybrid_clock.cc:648] HybridClock initialized: now 1786115456205558 us; error 30 us; skew 500 ppm
Aug 07 15:10:56 dist-test-slave-5bn2 krb5kdc[18416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:56.208137 18436 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:56.208564 18436 webserver.cc:460] Webserver started at http://127.0.0.1:34249/ using document root <none> and password file <none>
I20260807 15:10:56.208781 18436 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:56.208827 18436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:56.208951 18436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:56.210017 18436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "7c94b46d5194405eacb6fdbca4d70b56"
format_stamp: "Formatted at 2026-08-07 15:10:56 on dist-test-slave-5bn2"
server_key: "18013e08fce7d105f47b8390ccaf0894"
server_key_iv: "f8b5c396f5613f37f2378d46a9fb9636"
server_key_version: "encryptionkey@0"
I20260807 15:10:56.210397 18436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "7c94b46d5194405eacb6fdbca4d70b56"
format_stamp: "Formatted at 2026-08-07 15:10:56 on dist-test-slave-5bn2"
server_key: "18013e08fce7d105f47b8390ccaf0894"
server_key_iv: "f8b5c396f5613f37f2378d46a9fb9636"
server_key_version: "encryptionkey@0"
I20260807 15:10:56.211746 18436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:56.212455 18457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:56.212630 18436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:56.212719 18436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "7c94b46d5194405eacb6fdbca4d70b56"
format_stamp: "Formatted at 2026-08-07 15:10:56 on dist-test-slave-5bn2"
server_key: "18013e08fce7d105f47b8390ccaf0894"
server_key_iv: "f8b5c396f5613f37f2378d46a9fb9636"
server_key_version: "encryptionkey@0"
I20260807 15:10:56.212780 18436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:56.216109 18436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:56.217974 18436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:56.218083 18436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:56.221964 18436 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39351
I20260807 15:10:56.221994 18542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39351 every 8 connection(s)
I20260807 15:10:56.222333 18436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:56.223341 18543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:56.225659 18543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56: Bootstrap starting.
I20260807 15:10:56.226260 18543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:56.226562 18543 log.cc:824] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:56.227095 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 18436
I20260807 15:10:56.227209 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:56.227397 18948 external_mini_cluster.cc:1428] Setting key 322b1422d6cdfb2fde51a9bae68522be
I20260807 15:10:56.227399 18543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56: No bootstrap required, opened a new log
I20260807 15:10:56.229249 18543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c94b46d5194405eacb6fdbca4d70b56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39351 } }
I20260807 15:10:56.229400 18543 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:56.229439 18543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c94b46d5194405eacb6fdbca4d70b56, State: Initialized, Role: FOLLOWER
I20260807 15:10:56.229631 18543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c94b46d5194405eacb6fdbca4d70b56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39351 } }
I20260807 15:10:56.229733 18543 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:56.229770 18543 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:56.229822 18543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:56.230623 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:56.227927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:36749 (local address 127.0.0.1:39351)
0807 15:10:56.228117 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:56.228125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:56.228149 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:10:56.229222 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:56.229229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:56.229257 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:56.229376 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:56.230190 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:56.230193 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:56.230193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:56.230197 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:56.230237 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:56.230249 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:56.230301 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:56.230374 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:10:56.230411 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
I20260807 15:10:56.231056 18543 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c94b46d5194405eacb6fdbca4d70b56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39351 } }
I20260807 15:10:56.231168 18543 leader_election.cc:302] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c94b46d5194405eacb6fdbca4d70b56; no voters:
I20260807 15:10:56.231325 18543 leader_election.cc:288] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:56.231346 18550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:56.231503 18550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [term 1 LEADER]: Becoming Leader. State: Replica: 7c94b46d5194405eacb6fdbca4d70b56, State: Running, Role: LEADER
I20260807 15:10:56.231540 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:56.231582 18543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:56.231587 18550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c94b46d5194405eacb6fdbca4d70b56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39351 } }
I20260807 15:10:56.231958 18550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c94b46d5194405eacb6fdbca4d70b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c94b46d5194405eacb6fdbca4d70b56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39351 } } }
I20260807 15:10:56.232049 18550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:56.232167 18550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c94b46d5194405eacb6fdbca4d70b56. Latest consensus state: current_term: 1 leader_uuid: "7c94b46d5194405eacb6fdbca4d70b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c94b46d5194405eacb6fdbca4d70b56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39351 } } }
I20260807 15:10:56.232223 18550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:56.232342 18558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:56.232863 18558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:56.234346 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:56.233271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:45475 (local address 127.0.0.1:39351)
0807 15:10:56.233301 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:56.233303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:56.233433 (+ 130us) server_negotiation.cc:407] Connection header received
0807 15:10:56.233473 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:56.233474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:56.233480 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:56.233552 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:56.234136 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:56.234138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:56.234138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:56.234143 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:56.234157 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:56.234161 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:56.234222 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:56.234276 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:10:56.234289 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260807 15:10:56.235033 18558 catalog_manager.cc:1347] Generated new cluster ID: 818400da94684ad59a141ca5f5724c41
I20260807 15:10:56.235078 18558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:56.244630 18558 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:56.244812 18558 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:56.252053 18558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c94b46d5194405eacb6fdbca4d70b56: Generated new TSK 0
I20260807 15:10:56.252310 18558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:56.259178 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 18436
2026-08-07T15:10:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:56 dist-test-slave-5bn2 krb5kdc[18584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:56 dist-test-slave-5bn2 krb5kdc[18584](info): set up 2 sockets
Aug 07 15:10:56 dist-test-slave-5bn2 krb5kdc[18584](info): commencing operation
krb5kdc: starting...
W20260807 15:10:57.397570 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:57 dist-test-slave-5bn2 krb5kdc[18584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:57.519610 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:57.602370 18601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:57.602599 18601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:57.602632 18601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:57.603828 18601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:57.603874 18601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:57.603892 18601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:57.603909 18601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:57.603922 18601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:57.605495 18601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:57.605767 18601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:57.605983 18601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:57.610924 18609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:57.610926 18608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:57.611030 18611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:57.611125 18601 server_base.cc:1034] running on GCE node
I20260807 15:10:57.611452 18601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:57.611743 18601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:57.612887 18601 hybrid_clock.cc:648] HybridClock initialized: now 1786115457612866 us; error 36 us; skew 500 ppm
Aug 07 15:10:57 dist-test-slave-5bn2 krb5kdc[18584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115457, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:57.618690 18601 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:57.619239 18601 webserver.cc:460] Webserver started at http://127.0.0.1:44615/ using document root <none> and password file <none>
I20260807 15:10:57.619462 18601 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:57.619513 18601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:57.619650 18601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:57.620775 18601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95"
format_stamp: "Formatted at 2026-08-07 15:10:57 on dist-test-slave-5bn2"
server_key: "369c0d6a37f6c4426eee0644bbda0552"
server_key_iv: "736577d0dc3e5620e0aaf5db51e30710"
server_key_version: "encryptionkey@0"
I20260807 15:10:57.621132 18601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95"
format_stamp: "Formatted at 2026-08-07 15:10:57 on dist-test-slave-5bn2"
server_key: "369c0d6a37f6c4426eee0644bbda0552"
server_key_iv: "736577d0dc3e5620e0aaf5db51e30710"
server_key_version: "encryptionkey@0"
I20260807 15:10:57.622535 18601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:10:57.623296 18621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:57.623476 18601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:57.623569 18601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95"
format_stamp: "Formatted at 2026-08-07 15:10:57 on dist-test-slave-5bn2"
server_key: "369c0d6a37f6c4426eee0644bbda0552"
server_key_iv: "736577d0dc3e5620e0aaf5db51e30710"
server_key_version: "encryptionkey@0"
I20260807 15:10:57.623632 18601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:57.639523 18601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:57.641306 18601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:57.641463 18601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:57.645576 18601 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39967
I20260807 15:10:57.645620 18695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39967 every 8 connection(s)
I20260807 15:10:57.645952 18601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:57.646838 18696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:57.649186 18696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95: Bootstrap starting.
I20260807 15:10:57.649772 18696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:57.650046 18696 log.cc:824] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:57.650753 18696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95: No bootstrap required, opened a new log
I20260807 15:10:57.652714 18696 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260807 15:10:57.652865 18696 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:57.652887 18696 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a9fee9c8e64f8bb07dd4de2771ac95, State: Initialized, Role: FOLLOWER
I20260807 15:10:57.653036 18696 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260807 15:10:57.653111 18696 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:57.653133 18696 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:57.653179 18696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:57.653721 18696 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260807 15:10:57.653817 18696 leader_election.cc:302] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1a9fee9c8e64f8bb07dd4de2771ac95; no voters:
I20260807 15:10:57.653980 18696 leader_election.cc:288] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:57.654017 18700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:57.654183 18700 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [term 1 LEADER]: Becoming Leader. State: Replica: a1a9fee9c8e64f8bb07dd4de2771ac95, State: Running, Role: LEADER
I20260807 15:10:57.654224 18696 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:57.654277 18700 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } }
I20260807 15:10:57.654688 18702 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1a9fee9c8e64f8bb07dd4de2771ac95. Latest consensus state: current_term: 1 leader_uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } } }
I20260807 15:10:57.654767 18702 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:57.654829 18701 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a9fee9c8e64f8bb07dd4de2771ac95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39967 } } }
I20260807 15:10:57.655133 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 18601
I20260807 15:10:57.655258 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:57.655471 18948 external_mini_cluster.cc:1428] Setting key 1cb627401ddcee6844c42c6e91f02f78
I20260807 15:10:57.655670 18701 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95 [sys.catalog]: This master's current role is: LEADER
W20260807 15:10:57.655973 18718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:10:57.656101 18718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:10:57.656394 18710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:57.656613 18710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:57.658427 18710 catalog_manager.cc:1347] Generated new cluster ID: 436c854bc773426f816a905d25d08a0d
I20260807 15:10:57.658486 18710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:10:57 dist-test-slave-5bn2 krb5kdc[18584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:10:57.665676 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:57.655969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:59011 (local address 127.0.0.1:39967)
0807 15:10:57.656153 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:57.656162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:57.656293 (+ 131us) server_negotiation.cc:407] Connection header received
0807 15:10:57.656466 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:57.656473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:57.656535 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:57.656695 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:57.657470 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:57.658741 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:57.659538 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:57.659997 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:57.660106 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:57.662423 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:57.662427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:57.662427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:57.662440 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:57.664023 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:57.664529 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:57.664532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:57.664532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:57.664634 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:57.664842 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:57.664843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:57.664843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:57.665086 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:57.665249 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:57.665361 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:10:57.665463 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260807 15:10:57.666044 18710 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:57.666626 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:57.668267 18710 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:57.673924 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:57.668173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:33971 (local address 127.0.0.1:39967)
0807 15:10:57.668208 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:57.668210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:57.668384 (+ 174us) server_negotiation.cc:407] Connection header received
0807 15:10:57.668443 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:57.668443 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:57.668454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:57.668498 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:57.668998 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:57.669511 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:57.670244 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:57.670464 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:57.670517 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:57.671659 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:57.671660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:57.671660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:57.671667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:57.672900 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:57.673234 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:57.673235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:57.673235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:57.673285 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:57.673473 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:57.673473 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:57.673474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:57.673573 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:57.673677 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:57.673778 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 15:10:57.673838 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 15:10:57.677965 18710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1a9fee9c8e64f8bb07dd4de2771ac95: Generated new TSK 0
I20260807 15:10:57.678169 18710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:57.680459 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 18601
W20260807 15:10:57.698118 18581 connection.cc:535] client connection to 127.0.0.1:39967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39967): Connection reset by peer (error 104)
2026-08-07T15:10:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:57 dist-test-slave-5bn2 krb5kdc[18736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:57 dist-test-slave-5bn2 krb5kdc[18736](info): set up 2 sockets
Aug 07 15:10:57 dist-test-slave-5bn2 krb5kdc[18736](info): commencing operation
krb5kdc: starting...
W20260807 15:10:58.829036 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:10:58 dist-test-slave-5bn2 krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:10:58.947835 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:59.030455 18749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:59.030671 18749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:59.030702 18749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:59.031877 18749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:59.031925 18749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:59.031944 18749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:59.031960 18749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:59.031975 18749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:59.033455 18749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:10:59.033717 18749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:59.033924 18749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:59.039203 18758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:59.039247 18749 server_base.cc:1034] running on GCE node
W20260807 15:10:59.039192 18761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:59.039204 18759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:59.039737 18749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:59.040025 18749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:59.041167 18749 hybrid_clock.cc:648] HybridClock initialized: now 1786115459041153 us; error 36 us; skew 500 ppm
Aug 07 15:10:59 dist-test-slave-5bn2 krb5kdc[18736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115459, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:59.043617 18749 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:10:59.044054 18749 webserver.cc:460] Webserver started at http://127.0.0.1:43431/ using document root <none> and password file <none>
I20260807 15:10:59.044270 18749 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:59.044319 18749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:59.044445 18749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:59.045567 18749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "3382598699d346499d9c5c48a9bafbb6"
format_stamp: "Formatted at 2026-08-07 15:10:59 on dist-test-slave-5bn2"
server_key: "51b941a71fe0783aacebc9599fd9bd7f"
server_key_iv: "ab186b702455998f608f90f426feffb0"
server_key_version: "encryptionkey@0"
I20260807 15:10:59.045908 18749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "3382598699d346499d9c5c48a9bafbb6"
format_stamp: "Formatted at 2026-08-07 15:10:59 on dist-test-slave-5bn2"
server_key: "51b941a71fe0783aacebc9599fd9bd7f"
server_key_iv: "ab186b702455998f608f90f426feffb0"
server_key_version: "encryptionkey@0"
I20260807 15:10:59.047192 18749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:59.047847 18768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:59.048002 18749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:10:59.048096 18749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "3382598699d346499d9c5c48a9bafbb6"
format_stamp: "Formatted at 2026-08-07 15:10:59 on dist-test-slave-5bn2"
server_key: "51b941a71fe0783aacebc9599fd9bd7f"
server_key_iv: "ab186b702455998f608f90f426feffb0"
server_key_version: "encryptionkey@0"
I20260807 15:10:59.048157 18749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:59.067453 18749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:59.069248 18749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:59.069389 18749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:59.073486 18749 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37553
I20260807 15:10:59.073529 18846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37553 every 8 connection(s)
I20260807 15:10:59.073860 18749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:59.074713 18847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:59.077091 18847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6: Bootstrap starting.
I20260807 15:10:59.077680 18847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:59.077947 18847 log.cc:824] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:59.078611 18847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6: No bootstrap required, opened a new log
I20260807 15:10:59.080392 18847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3382598699d346499d9c5c48a9bafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37553 } }
I20260807 15:10:59.080536 18847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:59.080559 18847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3382598699d346499d9c5c48a9bafbb6, State: Initialized, Role: FOLLOWER
I20260807 15:10:59.080703 18847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3382598699d346499d9c5c48a9bafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37553 } }
I20260807 15:10:59.080783 18847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:59.080821 18847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:59.080868 18847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:59.081421 18847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3382598699d346499d9c5c48a9bafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37553 } }
I20260807 15:10:59.081532 18847 leader_election.cc:302] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3382598699d346499d9c5c48a9bafbb6; no voters:
I20260807 15:10:59.081679 18847 leader_election.cc:288] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:59.081738 18850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:59.081908 18850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [term 1 LEADER]: Becoming Leader. State: Replica: 3382598699d346499d9c5c48a9bafbb6, State: Running, Role: LEADER
I20260807 15:10:59.081915 18847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:59.082077 18850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3382598699d346499d9c5c48a9bafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37553 } }
I20260807 15:10:59.082518 18851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3382598699d346499d9c5c48a9bafbb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3382598699d346499d9c5c48a9bafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37553 } } }
I20260807 15:10:59.082669 18851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:59.082684 18853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3382598699d346499d9c5c48a9bafbb6. Latest consensus state: current_term: 1 leader_uuid: "3382598699d346499d9c5c48a9bafbb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3382598699d346499d9c5c48a9bafbb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37553 } } }
I20260807 15:10:59.082842 18853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:59.083133 18862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:59.083446 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 18749
I20260807 15:10:59.083572 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:10:59.083655 18862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:59.083794 18948 external_mini_cluster.cc:1428] Setting key 7b936b8d35ca521086c1e373b5f39755
I20260807 15:10:59.085770 18862 catalog_manager.cc:1347] Generated new cluster ID: 594986b2a2dd42b8bdd22918b8ff5ac8
I20260807 15:10:59.085829 18862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:59.089224 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:59.084243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:38645 (local address 127.0.0.1:37553)
0807 15:10:59.084961 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:59.084969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:59.085001 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:10:59.085128 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:59.085136 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:59.085215 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:59.085349 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:59.085594 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:59.086571 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:59.087435 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:59.087879 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:59.087975 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:59.088632 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:59.088636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:59.088636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:59.088801 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:59.088832 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:59.088843 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:59.088909 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:59.088998 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 15:10:59.089026 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":90,"threads_started":1}
I20260807 15:10:59.090062 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:10:59.097844 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:59.095163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:51293 (local address 127.0.0.1:37553)
0807 15:10:59.095206 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:59.095207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:59.095327 (+ 120us) server_negotiation.cc:407] Connection header received
0807 15:10:59.095375 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:59.095375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:59.095386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:59.095438 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:59.095560 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:59.096050 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:59.096706 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:59.096924 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:59.096976 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:59.097602 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:59.097603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:59.097604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:59.097608 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:59.097623 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:59.097627 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:59.097670 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:59.097744 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 15:10:59.097758 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:10:59.101357 18862 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:59.102018 18862 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:59.104501 18862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3382598699d346499d9c5c48a9bafbb6: Generated new TSK 0
I20260807 15:10:59.104667 18862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:59.109889 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 18749
W20260807 15:10:59.125554 18734 connection.cc:535] client connection to 127.0.0.1:37553 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37553): Connection reset by peer (error 104)
2026-08-07T15:10:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:59 dist-test-slave-5bn2 krb5kdc[18895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:59 dist-test-slave-5bn2 krb5kdc[18895](info): set up 2 sockets
Aug 07 15:10:59 dist-test-slave-5bn2 krb5kdc[18895](info): commencing operation
krb5kdc: starting...
W20260807 15:11:00.256717 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:00 dist-test-slave-5bn2 krb5kdc[18895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:11:00.383697 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:00.467036 18912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:00.467252 18912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:00.467272 18912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:00.468439 18912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:00.468482 18912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:00.468494 18912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:00.468505 18912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:00.468528 18912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:00.470139 18912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:00.470386 18912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:00.470603 18912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:00.475839 18924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:00.475903 18912 server_base.cc:1034] running on GCE node
W20260807 15:11:00.475831 18920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:00.475831 18919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:00.476367 18912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:00.476727 18912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:00.477885 18912 hybrid_clock.cc:648] HybridClock initialized: now 1786115460477864 us; error 31 us; skew 500 ppm
Aug 07 15:11:00 dist-test-slave-5bn2 krb5kdc[18895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:00.480435 18912 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:11:00.480887 18912 webserver.cc:460] Webserver started at http://127.0.0.1:35143/ using document root <none> and password file <none>
I20260807 15:11:00.481096 18912 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:00.481143 18912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:00.481268 18912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:00.482321 18912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "84c264684d054439b5282f66947f3644"
format_stamp: "Formatted at 2026-08-07 15:11:00 on dist-test-slave-5bn2"
server_key: "35ea97761743d0290f360c50fe999318"
server_key_iv: "468a57839ff5503e26ecdf63cd88371a"
server_key_version: "encryptionkey@0"
I20260807 15:11:00.482661 18912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "84c264684d054439b5282f66947f3644"
format_stamp: "Formatted at 2026-08-07 15:11:00 on dist-test-slave-5bn2"
server_key: "35ea97761743d0290f360c50fe999318"
server_key_iv: "468a57839ff5503e26ecdf63cd88371a"
server_key_version: "encryptionkey@0"
I20260807 15:11:00.483976 18912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:00.484676 18932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:00.484831 18912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:00.484918 18912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "84c264684d054439b5282f66947f3644"
format_stamp: "Formatted at 2026-08-07 15:11:00 on dist-test-slave-5bn2"
server_key: "35ea97761743d0290f360c50fe999318"
server_key_iv: "468a57839ff5503e26ecdf63cd88371a"
server_key_version: "encryptionkey@0"
I20260807 15:11:00.484984 18912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:00.488315 18912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:00.490118 18912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:00.490247 18912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:00.494232 18912 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42883
I20260807 15:11:00.494287 19013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42883 every 8 connection(s)
I20260807 15:11:00.494774 18912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:00.495942 19014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:00.498208 19014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644: Bootstrap starting.
I20260807 15:11:00.498775 19014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:00.499132 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 18912
I20260807 15:11:00.499229 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:00.499384 18948 external_mini_cluster.cc:1428] Setting key 1fc0bd5c3d69fa03251c267ad4b3b932
I20260807 15:11:00.499657 19014 log.cc:824] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:00.500550 19014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644: No bootstrap required, opened a new log
I20260807 15:11:00.502014 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:00.499837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:55179 (local address 127.0.0.1:42883)
0807 15:11:00.500062 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:00.500072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:00.500107 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:11:00.500748 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:00.500758 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:00.500793 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:00.500961 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:00.501565 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:00.501568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:00.501569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:00.501573 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:00.501624 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:00.501638 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:00.501727 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:00.501756 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 15:11:00.501799 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260807 15:11:00.502967 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:00.502835 19014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c264684d054439b5282f66947f3644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } }
I20260807 15:11:00.503093 19014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:00.503136 19014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c264684d054439b5282f66947f3644, State: Initialized, Role: FOLLOWER
I20260807 15:11:00.503250 19014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c264684d054439b5282f66947f3644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } }
I20260807 15:11:00.503324 19014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:00.503352 19014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:00.503409 19014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:00.503984 19014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c264684d054439b5282f66947f3644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } }
I20260807 15:11:00.504093 19014 leader_election.cc:302] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84c264684d054439b5282f66947f3644; no voters:
I20260807 15:11:00.504274 19014 leader_election.cc:288] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:00.504317 19024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:00.504465 19024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [term 1 LEADER]: Becoming Leader. State: Replica: 84c264684d054439b5282f66947f3644, State: Running, Role: LEADER
I20260807 15:11:00.504525 19014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:00.504607 19024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c264684d054439b5282f66947f3644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } }
I20260807 15:11:00.505146 19028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84c264684d054439b5282f66947f3644. Latest consensus state: current_term: 1 leader_uuid: "84c264684d054439b5282f66947f3644" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c264684d054439b5282f66947f3644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } } }
I20260807 15:11:00.505244 19028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:00.505468 19036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:00.506032 19036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:00.506110 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:00.504509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.170:58289 (local address 127.0.0.1:42883)
0807 15:11:00.504566 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:00.504568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:00.504638 (+ 70us) server_negotiation.cc:407] Connection header received
0807 15:11:00.504715 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:00.504716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:00.504734 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:00.504782 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:00.505851 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:00.505852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:00.505852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:00.505858 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:00.505873 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:00.505876 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:00.505921 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:00.506037 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 15:11:00.506049 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:11:00.506182 19026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84c264684d054439b5282f66947f3644" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84c264684d054439b5282f66947f3644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } } }
I20260807 15:11:00.506249 19026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:00.507651 19036 catalog_manager.cc:1347] Generated new cluster ID: e0ae433ff46141c7b2241bc1df7c621e
I20260807 15:11:00.507696 19036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:00.527704 19036 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:00.527892 19036 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:00.535614 19036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84c264684d054439b5282f66947f3644: Generated new TSK 0
I20260807 15:11:00.535802 19036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:00.542057 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 18912
2026-08-07T15:11:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:00 dist-test-slave-5bn2 krb5kdc[19058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:00 dist-test-slave-5bn2 krb5kdc[19058](info): set up 2 sockets
Aug 07 15:11:00 dist-test-slave-5bn2 krb5kdc[19058](info): commencing operation
krb5kdc: starting...
W20260807 15:11:01.677989 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:01 dist-test-slave-5bn2 krb5kdc[19058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:11:01.801894 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39653
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:39653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:01.883916 19076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:01.884299 19076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:01.884378 19076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:01.885828 19076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:01.885890 19076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:01.885910 19076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:01.885937 19076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:01.885960 19076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:01.888008 19076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:39653
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:39653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:01.888284 19076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:01.888484 19076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:01.893296 19086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:01.893428 19076 server_base.cc:1034] running on GCE node
W20260807 15:11:01.893527 19084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:01.893296 19083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:01.893812 19076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:01.894079 19076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:01.895221 19076 hybrid_clock.cc:648] HybridClock initialized: now 1786115461895201 us; error 33 us; skew 500 ppm
Aug 07 15:11:01 dist-test-slave-5bn2 krb5kdc[19058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:01.897938 19076 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:11:01.898346 19076 webserver.cc:460] Webserver started at http://127.18.129.62:41765/ using document root <none> and password file <none>
I20260807 15:11:01.898533 19076 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:01.898574 19076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:01.898663 19076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:01.899741 19076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "90d220cc0b9046a9926ffe4df9a48eac"
format_stamp: "Formatted at 2026-08-07 15:11:01 on dist-test-slave-5bn2"
server_key: "368b98f47c22825a619b98de09578108"
server_key_iv: "16862176fa28f9b8d88ddeecee4bd3e6"
server_key_version: "encryptionkey@0"
I20260807 15:11:01.900051 19076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "90d220cc0b9046a9926ffe4df9a48eac"
format_stamp: "Formatted at 2026-08-07 15:11:01 on dist-test-slave-5bn2"
server_key: "368b98f47c22825a619b98de09578108"
server_key_iv: "16862176fa28f9b8d88ddeecee4bd3e6"
server_key_version: "encryptionkey@0"
I20260807 15:11:01.901319 19076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:11:01.902151 19098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:01.902297 19076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:01.902386 19076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "90d220cc0b9046a9926ffe4df9a48eac"
format_stamp: "Formatted at 2026-08-07 15:11:01 on dist-test-slave-5bn2"
server_key: "368b98f47c22825a619b98de09578108"
server_key_iv: "16862176fa28f9b8d88ddeecee4bd3e6"
server_key_version: "encryptionkey@0"
I20260807 15:11:01.902458 19076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:01.922819 19076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:01.924654 19076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:01.924767 19076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:01.928684 19076 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:39653
I20260807 15:11:01.928726 19161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:39653 every 8 connection(s)
I20260807 15:11:01.929064 19076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:01.929972 19162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:01.932377 19162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac: Bootstrap starting.
I20260807 15:11:01.932973 19162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:01.933259 19162 log.cc:824] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:01.933965 19162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac: No bootstrap required, opened a new log
I20260807 15:11:01.935706 19162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d220cc0b9046a9926ffe4df9a48eac" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39653 } }
I20260807 15:11:01.935855 19162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:01.935889 19162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d220cc0b9046a9926ffe4df9a48eac, State: Initialized, Role: FOLLOWER
I20260807 15:11:01.936022 19162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d220cc0b9046a9926ffe4df9a48eac" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39653 } }
I20260807 15:11:01.936085 19162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:01.936115 19162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:01.936167 19162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:01.937076 19162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d220cc0b9046a9926ffe4df9a48eac" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39653 } }
I20260807 15:11:01.937155 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19076
I20260807 15:11:01.937182 19162 leader_election.cc:302] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90d220cc0b9046a9926ffe4df9a48eac; no voters:
I20260807 15:11:01.937263 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:01.937373 19162 leader_election.cc:288] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:01.937412 19166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:01.937451 18948 external_mini_cluster.cc:1428] Setting key 1ca1b2de5608a8704bb1b2f4237dab22
I20260807 15:11:01.937534 19166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [term 1 LEADER]: Becoming Leader. State: Replica: 90d220cc0b9046a9926ffe4df9a48eac, State: Running, Role: LEADER
I20260807 15:11:01.937613 19166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d220cc0b9046a9926ffe4df9a48eac" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39653 } }
I20260807 15:11:01.937665 19162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:01.938297 19167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90d220cc0b9046a9926ffe4df9a48eac. Latest consensus state: current_term: 1 leader_uuid: "90d220cc0b9046a9926ffe4df9a48eac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d220cc0b9046a9926ffe4df9a48eac" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39653 } } }
I20260807 15:11:01.938310 19168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90d220cc0b9046a9926ffe4df9a48eac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90d220cc0b9046a9926ffe4df9a48eac" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 39653 } } }
I20260807 15:11:01.938397 19167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:01.938413 19168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:01.939067 19182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:01.939601 19182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:01.943288 19182 catalog_manager.cc:1347] Generated new cluster ID: e2a7f5efbbdd484e890529de6bca2ec3
I20260807 15:11:01.943336 19182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:01.948494 19182 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:01.948966 19182 catalog_manager.cc:1504] Loading token signing keys...
Aug 07 15:11:01 dist-test-slave-5bn2 krb5kdc[19058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:11:01.953346 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:01.937931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56747 (local address 127.18.129.62:39653)
0807 15:11:01.938250 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:01.938261 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:01.938298 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:11:01.938418 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:01.938425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:01.938487 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:01.938626 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:01.939554 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:01.940793 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:01.941539 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:01.941987 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:01.942080 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:01.949602 (+ 7522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:01.949606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:01.949607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:01.949620 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:01.951577 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:01.952036 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:01.952039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:01.952039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:01.952142 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:01.952346 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:01.952347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:01.952347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:01.952691 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:01.952833 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:01.952992 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:11:01.953088 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":70,"threads_started":1}
I20260807 15:11:01.954272 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:01.956770 19182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90d220cc0b9046a9926ffe4df9a48eac: Generated new TSK 0
I20260807 15:11:01.956972 19182 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:01.962512 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:01.955879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57401 (local address 127.18.129.62:39653)
0807 15:11:01.955913 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:01.955915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:01.955949 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:11:01.956020 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:01.956021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:01.956029 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:01.956079 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:01.956694 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:01.957650 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:01.958684 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:01.958925 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:01.958983 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:01.960266 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:01.960268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:01.960268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:01.960273 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:01.961487 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:01.961868 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:01.961869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:01.961869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:01.961916 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:01.962115 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:01.962116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:01.962116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:01.962202 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:01.962273 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:01.962405 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 15:11:01.962449 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:11:01.964401 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19076
W20260807 15:11:01.980397 19056 connection.cc:535] client connection to 127.18.129.62:39653 recv error: Network error: failed to read from TLS socket (remote: 127.18.129.62:39653): Connection reset by peer (error 104)
2026-08-07T15:11:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:02 dist-test-slave-5bn2 krb5kdc[19212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:02 dist-test-slave-5bn2 krb5kdc[19212](info): set up 2 sockets
Aug 07 15:11:02 dist-test-slave-5bn2 krb5kdc[19212](info): commencing operation
krb5kdc: starting...
W20260807 15:11:03.102816 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:03 dist-test-slave-5bn2 krb5kdc[19212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:11:03.223901 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:35511
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:35511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:03.305802 19230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:03.306025 19230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:03.306058 19230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:03.307231 19230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:03.307274 19230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:03.307292 19230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:03.307307 19230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:03.307322 19230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:03.308877 19230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:35511
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:35511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:03.309139 19230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:03.309353 19230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:03.314332 19242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:03.314306 19239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:03.314294 19236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:03.314388 19230 server_base.cc:1034] running on GCE node
I20260807 15:11:03.314764 19230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:03.315083 19230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:03.316268 19230 hybrid_clock.cc:648] HybridClock initialized: now 1786115463316250 us; error 29 us; skew 500 ppm
Aug 07 15:11:03 dist-test-slave-5bn2 krb5kdc[19212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115463, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:03.322875 19230 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:11:03.323311 19230 webserver.cc:460] Webserver started at http://127.18.129.62:39943/ using document root <none> and password file <none>
I20260807 15:11:03.323520 19230 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:03.323570 19230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:03.323705 19230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:03.324783 19230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "4181a9a8463140bfbc5a3d99db2e564a"
format_stamp: "Formatted at 2026-08-07 15:11:03 on dist-test-slave-5bn2"
server_key: "31c3f12b9466063131a0efa0227c4092"
server_key_iv: "9e95001f3ad9e1d53a33ec29787a043d"
server_key_version: "encryptionkey@0"
I20260807 15:11:03.325141 19230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "4181a9a8463140bfbc5a3d99db2e564a"
format_stamp: "Formatted at 2026-08-07 15:11:03 on dist-test-slave-5bn2"
server_key: "31c3f12b9466063131a0efa0227c4092"
server_key_iv: "9e95001f3ad9e1d53a33ec29787a043d"
server_key_version: "encryptionkey@0"
I20260807 15:11:03.326525 19230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:03.327324 19253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:03.327500 19230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:03.327607 19230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "4181a9a8463140bfbc5a3d99db2e564a"
format_stamp: "Formatted at 2026-08-07 15:11:03 on dist-test-slave-5bn2"
server_key: "31c3f12b9466063131a0efa0227c4092"
server_key_iv: "9e95001f3ad9e1d53a33ec29787a043d"
server_key_version: "encryptionkey@0"
I20260807 15:11:03.327675 19230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:03.354681 19230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:03.356591 19230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:03.356734 19230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:03.360831 19230 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:35511
I20260807 15:11:03.360877 19322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:35511 every 8 connection(s)
I20260807 15:11:03.361215 19230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:03.362099 19323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:03.364363 19323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a: Bootstrap starting.
I20260807 15:11:03.364892 19323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:03.365126 19323 log.cc:824] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:03.365751 19323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a: No bootstrap required, opened a new log
I20260807 15:11:03.367358 19323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181a9a8463140bfbc5a3d99db2e564a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35511 } }
I20260807 15:11:03.367497 19323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:03.367518 19323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4181a9a8463140bfbc5a3d99db2e564a, State: Initialized, Role: FOLLOWER
I20260807 15:11:03.367630 19323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181a9a8463140bfbc5a3d99db2e564a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35511 } }
I20260807 15:11:03.367707 19323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:03.367738 19323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:03.367771 19323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:03.368533 19323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181a9a8463140bfbc5a3d99db2e564a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35511 } }
I20260807 15:11:03.368618 19323 leader_election.cc:302] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4181a9a8463140bfbc5a3d99db2e564a; no voters:
I20260807 15:11:03.368741 19323 leader_election.cc:288] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:03.368821 19326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:03.368957 19323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:03.369025 19326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [term 1 LEADER]: Becoming Leader. State: Replica: 4181a9a8463140bfbc5a3d99db2e564a, State: Running, Role: LEADER
I20260807 15:11:03.369120 19326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181a9a8463140bfbc5a3d99db2e564a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35511 } }
I20260807 15:11:03.369565 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19230
I20260807 15:11:03.369545 19328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4181a9a8463140bfbc5a3d99db2e564a. Latest consensus state: current_term: 1 leader_uuid: "4181a9a8463140bfbc5a3d99db2e564a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181a9a8463140bfbc5a3d99db2e564a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35511 } } }
I20260807 15:11:03.369642 19328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:03.369699 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:03.369539 19327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4181a9a8463140bfbc5a3d99db2e564a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4181a9a8463140bfbc5a3d99db2e564a" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35511 } } }
I20260807 15:11:03.369815 19327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:03.369907 18948 external_mini_cluster.cc:1428] Setting key 1be9db01be4c2c1b1b8ac58a08566ab8
W20260807 15:11:03.370934 19347 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:11:03.370981 19347 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:03.371037 19339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:03.371210 19339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:03.372684 19339 catalog_manager.cc:1347] Generated new cluster ID: 4e847b5efa674177b477e57b8a4eb23f
I20260807 15:11:03.372732 19339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:03.376102 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:03.370358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41785 (local address 127.18.129.62:35511)
0807 15:11:03.371067 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:03.371077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:03.371109 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:11:03.371232 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:03.371240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:03.371312 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:03.371449 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:03.371689 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:03.372869 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:03.373937 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:03.374516 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:03.374604 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:03.375330 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:03.375334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:03.375334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:03.375541 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:03.375578 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:03.375591 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:03.375663 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:03.375757 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 15:11:03.375881 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":74,"threads_started":1}
I20260807 15:11:03.377274 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:03.380749 19339 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:03.381412 19339 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:03.382140 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:03.378677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53813 (local address 127.18.129.62:35511)
0807 15:11:03.378710 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:03.378712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:03.378836 (+ 124us) server_negotiation.cc:407] Connection header received
0807 15:11:03.378936 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:03.378937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:03.378949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:03.378978 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:03.379189 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:03.379924 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:03.380814 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:03.381111 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:03.381149 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:03.381907 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:03.381909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:03.381909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:03.381913 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:03.381930 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:03.381935 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:03.381995 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:03.382053 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 15:11:03.382064 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:11:03.386669 19339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4181a9a8463140bfbc5a3d99db2e564a: Generated new TSK 0
I20260807 15:11:03.386819 19339 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:03.391983 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19230
W20260807 15:11:03.407943 19205 connection.cc:535] client connection to 127.18.129.62:35511 recv error: Network error: failed to read from TLS socket (remote: 127.18.129.62:35511): Connection reset by peer (error 104)
2026-08-07T15:11:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:03 dist-test-slave-5bn2 krb5kdc[19368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:03 dist-test-slave-5bn2 krb5kdc[19368](info): set up 2 sockets
Aug 07 15:11:03 dist-test-slave-5bn2 krb5kdc[19368](info): commencing operation
krb5kdc: starting...
W20260807 15:11:04.558880 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:04 dist-test-slave-5bn2 krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:11:04.679797 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46621
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:04.762401 19385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:04.762621 19385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:04.762652 19385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:04.763801 19385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:04.763844 19385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:04.763862 19385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:04.763878 19385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:04.763892 19385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:04.765338 19385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46621
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:04.765596 19385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:04.765806 19385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:04.770756 19393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:04.770756 19397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:04.770931 19385 server_base.cc:1034] running on GCE node
W20260807 15:11:04.770766 19394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:04.771278 19385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:04.771543 19385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:04.772677 19385 hybrid_clock.cc:648] HybridClock initialized: now 1786115464772665 us; error 32 us; skew 500 ppm
Aug 07 15:11:04 dist-test-slave-5bn2 krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115464, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:04.775094 19385 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:11:04.775516 19385 webserver.cc:460] Webserver started at http://127.18.129.62:46455/ using document root <none> and password file <none>
I20260807 15:11:04.775715 19385 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:04.775763 19385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:04.775887 19385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:04.776968 19385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "b20a8bbbf5e6431395a0bd7143ded50e"
format_stamp: "Formatted at 2026-08-07 15:11:04 on dist-test-slave-5bn2"
server_key: "887d6567c3f1b0daaa6bba535fd75d30"
server_key_iv: "fb559a559e4e80aff34ab00cf57c50d2"
server_key_version: "encryptionkey@0"
I20260807 15:11:04.777314 19385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "b20a8bbbf5e6431395a0bd7143ded50e"
format_stamp: "Formatted at 2026-08-07 15:11:04 on dist-test-slave-5bn2"
server_key: "887d6567c3f1b0daaa6bba535fd75d30"
server_key_iv: "fb559a559e4e80aff34ab00cf57c50d2"
server_key_version: "encryptionkey@0"
I20260807 15:11:04.778614 19385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:04.779340 19409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:04.779486 19385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:04.779573 19385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "b20a8bbbf5e6431395a0bd7143ded50e"
format_stamp: "Formatted at 2026-08-07 15:11:04 on dist-test-slave-5bn2"
server_key: "887d6567c3f1b0daaa6bba535fd75d30"
server_key_iv: "fb559a559e4e80aff34ab00cf57c50d2"
server_key_version: "encryptionkey@0"
I20260807 15:11:04.779640 19385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:04.782874 19385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:04.784935 19385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:04.785065 19385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:04.788910 19385 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:46621
I20260807 15:11:04.788942 19488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:46621 every 8 connection(s)
I20260807 15:11:04.789252 19385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:04.790230 19490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:04.792543 19490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e: Bootstrap starting.
I20260807 15:11:04.793093 19490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:04.793357 19490 log.cc:824] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:04.794013 19490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e: No bootstrap required, opened a new log
I20260807 15:11:04.795172 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19385
I20260807 15:11:04.795300 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:04.795487 18948 external_mini_cluster.cc:1428] Setting key a2574f4de9db9af08041907975fd771a
I20260807 15:11:04.796001 19490 raft_consensus.cc:348] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20a8bbbf5e6431395a0bd7143ded50e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46621 } }
I20260807 15:11:04.796185 19490 raft_consensus.cc:374] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:04.796236 19490 raft_consensus.cc:729] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b20a8bbbf5e6431395a0bd7143ded50e, State: Initialized, Role: FOLLOWER
I20260807 15:11:04.796361 19490 consensus_queue.cc:260] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20a8bbbf5e6431395a0bd7143ded50e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46621 } }
I20260807 15:11:04.796433 19490 raft_consensus.cc:388] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:04.796455 19490 raft_consensus.cc:482] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:04.796512 19490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:04.797595 19490 raft_consensus.cc:504] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20a8bbbf5e6431395a0bd7143ded50e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46621 } }
I20260807 15:11:04.797714 19490 leader_election.cc:302] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b20a8bbbf5e6431395a0bd7143ded50e; no voters:
I20260807 15:11:04.797724 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:04.796030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44165 (local address 127.18.129.62:46621)
0807 15:11:04.796203 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:04.796212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:04.796267 (+ 55us) server_negotiation.cc:407] Connection header received
0807 15:11:04.796391 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:04.796399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:04.796443 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:04.796587 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:04.797206 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:04.797209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:04.797210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:04.797213 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:04.797254 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:04.797266 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:04.797318 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:04.797404 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:11:04.797491 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260807 15:11:04.797904 19490 leader_election.cc:288] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:04.797945 19496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:04.798120 19496 raft_consensus.cc:686] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [term 1 LEADER]: Becoming Leader. State: Replica: b20a8bbbf5e6431395a0bd7143ded50e, State: Running, Role: LEADER
I20260807 15:11:04.798152 19490 sys_catalog.cc:564] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:04.798209 19496 consensus_queue.cc:237] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20a8bbbf5e6431395a0bd7143ded50e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46621 } }
I20260807 15:11:04.798619 19498 sys_catalog.cc:455] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b20a8bbbf5e6431395a0bd7143ded50e. Latest consensus state: current_term: 1 leader_uuid: "b20a8bbbf5e6431395a0bd7143ded50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20a8bbbf5e6431395a0bd7143ded50e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46621 } } }
I20260807 15:11:04.798708 19498 sys_catalog.cc:458] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:04.798632 19497 sys_catalog.cc:455] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b20a8bbbf5e6431395a0bd7143ded50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20a8bbbf5e6431395a0bd7143ded50e" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46621 } } }
I20260807 15:11:04.798754 19497 sys_catalog.cc:458] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:04.798918 19506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:04.799129 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:04.800129 19506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:04.801704 19506 catalog_manager.cc:1347] Generated new cluster ID: 5041897193bb4b6aa1acf532d1f46c71
I20260807 15:11:04.801759 19506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:04.802193 19495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:04.801021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47299 (local address 127.18.129.62:46621)
0807 15:11:04.801057 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:04.801059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:04.801230 (+ 171us) server_negotiation.cc:407] Connection header received
0807 15:11:04.801289 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:04.801290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:04.801297 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:04.801354 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:04.801995 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:04.801997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:04.801997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:04.802001 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:04.802017 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:04.802022 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:04.802068 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:04.802117 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 15:11:04.802131 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:11:04.815387 19506 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:04.815547 19506 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:04.823778 19506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b20a8bbbf5e6431395a0bd7143ded50e: Generated new TSK 0
I20260807 15:11:04.824009 19506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:04.832274 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19385
2026-08-07T15:11:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:04 dist-test-slave-5bn2 krb5kdc[19526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:04 dist-test-slave-5bn2 krb5kdc[19526](info): set up 2 sockets
Aug 07 15:11:04 dist-test-slave-5bn2 krb5kdc[19526](info): commencing operation
krb5kdc: starting...
W20260807 15:11:05.976516 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:06 dist-test-slave-5bn2 krb5kdc[19526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:11:06.098404 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:36351
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:36351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:06.182011 19544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:06.182230 19544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:06.182253 19544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:06.183444 19544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:06.183486 19544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:06.183499 19544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:06.183511 19544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:06.183521 19544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:06.184983 19544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:36351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:36351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:06.185204 19544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:06.185415 19544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:06.190284 19551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:06.190294 19554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:06.190338 19544 server_base.cc:1034] running on GCE node
W20260807 15:11:06.190361 19552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:06.190791 19544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:06.191095 19544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:06.192241 19544 hybrid_clock.cc:648] HybridClock initialized: now 1786115466192229 us; error 29 us; skew 500 ppm
Aug 07 15:11:06 dist-test-slave-5bn2 krb5kdc[19526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115466, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:06.194731 19544 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:11:06.195192 19544 webserver.cc:460] Webserver started at http://127.18.129.62:34587/ using document root <none> and password file <none>
I20260807 15:11:06.195369 19544 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:06.195408 19544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:06.195497 19544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:06.196535 19544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "9a7dc254aff74877b493b96ba3917398"
format_stamp: "Formatted at 2026-08-07 15:11:06 on dist-test-slave-5bn2"
server_key: "130a20d58c506e978dadc23308386a36"
server_key_iv: "b45dbe85b52922a413513abfe146084f"
server_key_version: "encryptionkey@0"
I20260807 15:11:06.196892 19544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "9a7dc254aff74877b493b96ba3917398"
format_stamp: "Formatted at 2026-08-07 15:11:06 on dist-test-slave-5bn2"
server_key: "130a20d58c506e978dadc23308386a36"
server_key_iv: "b45dbe85b52922a413513abfe146084f"
server_key_version: "encryptionkey@0"
I20260807 15:11:06.198421 19544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:06.199553 19564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:06.199715 19544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:11:06.199805 19544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "9a7dc254aff74877b493b96ba3917398"
format_stamp: "Formatted at 2026-08-07 15:11:06 on dist-test-slave-5bn2"
server_key: "130a20d58c506e978dadc23308386a36"
server_key_iv: "b45dbe85b52922a413513abfe146084f"
server_key_version: "encryptionkey@0"
I20260807 15:11:06.199867 19544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:06.211257 19544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:06.213292 19544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:06.213398 19544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:06.217252 19544 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:36351
I20260807 15:11:06.217298 19642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:36351 every 8 connection(s)
I20260807 15:11:06.217613 19544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:06.218564 19643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:06.221083 19643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398: Bootstrap starting.
I20260807 15:11:06.221684 19643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:06.221985 19643 log.cc:824] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:06.222813 19643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398: No bootstrap required, opened a new log
I20260807 15:11:06.224062 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19544
I20260807 15:11:06.224157 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:06.224328 18948 external_mini_cluster.cc:1428] Setting key 39200affa67a44bda787e8192212401c
I20260807 15:11:06.224655 19643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a7dc254aff74877b493b96ba3917398" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36351 } }
I20260807 15:11:06.224834 19643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:06.224880 19643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a7dc254aff74877b493b96ba3917398, State: Initialized, Role: FOLLOWER
I20260807 15:11:06.225013 19643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a7dc254aff74877b493b96ba3917398" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36351 } }
I20260807 15:11:06.225096 19643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:06.225140 19643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:06.225186 19643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:06.225931 19643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a7dc254aff74877b493b96ba3917398" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36351 } }
I20260807 15:11:06.226047 19643 leader_election.cc:302] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a7dc254aff74877b493b96ba3917398; no voters:
I20260807 15:11:06.226205 19643 leader_election.cc:288] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:06.226243 19650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:06.226383 19650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [term 1 LEADER]: Becoming Leader. State: Replica: 9a7dc254aff74877b493b96ba3917398, State: Running, Role: LEADER
I20260807 15:11:06.226460 19643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:06.226476 19650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a7dc254aff74877b493b96ba3917398" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36351 } }
I20260807 15:11:06.226873 19650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a7dc254aff74877b493b96ba3917398. Latest consensus state: current_term: 1 leader_uuid: "9a7dc254aff74877b493b96ba3917398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a7dc254aff74877b493b96ba3917398" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36351 } } }
I20260807 15:11:06.226949 19650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:06.227109 19652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a7dc254aff74877b493b96ba3917398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a7dc254aff74877b493b96ba3917398" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 36351 } } }
I20260807 15:11:06.227183 19652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:06.227749 19659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:06.228268 19659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:06.229897 19659 catalog_manager.cc:1347] Generated new cluster ID: cacfb97624684ab9955ed4226a27413e
I20260807 15:11:06.229949 19659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:11:06 dist-test-slave-5bn2 krb5kdc[19526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:11:06.234352 19659 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:06.234993 19659 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:06.235587 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:06.224798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34781 (local address 127.18.129.62:36351)
0807 15:11:06.224996 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:06.225004 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:06.225037 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:11:06.225169 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:06.225177 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:06.225232 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:06.225392 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:06.226027 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:06.227512 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:06.228485 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:06.229054 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:06.229163 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:06.231959 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:06.231964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:06.231964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:06.231977 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:06.233886 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:06.234234 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:06.234237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:06.234238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:06.234375 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:06.234615 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:06.234616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:06.234616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:06.234915 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:06.235136 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:06.235207 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:11:06.235344 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260807 15:11:06.236464 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:06.244478 19659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a7dc254aff74877b493b96ba3917398: Generated new TSK 0
I20260807 15:11:06.244661 19659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:06.245039 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:06.238016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36499 (local address 127.18.129.62:36351)
0807 15:11:06.238058 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:06.238060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:06.238176 (+ 116us) server_negotiation.cc:407] Connection header received
0807 15:11:06.238237 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:06.238237 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:06.238247 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:06.238296 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:06.238911 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:06.239655 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:06.240674 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:06.240961 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:06.241014 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:06.242341 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:06.242342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:06.242343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:06.242349 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:06.243889 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:06.244257 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:06.244258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:06.244258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:06.244318 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:06.244507 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:06.244508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:06.244509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:06.244640 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:06.244725 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:06.244878 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 15:11:06.244956 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:11:06.246721 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19544
2026-08-07T15:11:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:06 dist-test-slave-5bn2 krb5kdc[19683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:06 dist-test-slave-5bn2 krb5kdc[19683](info): set up 2 sockets
Aug 07 15:11:06 dist-test-slave-5bn2 krb5kdc[19683](info): commencing operation
krb5kdc: starting...
W20260807 15:11:07.387450 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:07 dist-test-slave-5bn2 krb5kdc[19683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:11:07.507562 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:41143
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:38379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:41143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:07.590265 19702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:07.590481 19702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:07.590503 19702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:07.591698 19702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:07.591746 19702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:07.591759 19702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:07.591784 19702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:07.591807 19702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:07.593396 19702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:38379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:41143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:41143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:07.593652 19702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:07.593866 19702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:11:07.598874 19702 server_base.cc:1034] running on GCE node
W20260807 15:11:07.598803 19712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:07.598819 19714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:07.598803 19709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:07.599331 19702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:07.599627 19702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:07.600775 19702 hybrid_clock.cc:648] HybridClock initialized: now 1786115467600756 us; error 35 us; skew 500 ppm
Aug 07 15:11:07 dist-test-slave-5bn2 krb5kdc[19683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115467, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:07.603642 19702 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:11:07.604099 19702 webserver.cc:460] Webserver started at http://127.18.129.62:46805/ using document root <none> and password file <none>
I20260807 15:11:07.604317 19702 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:07.604367 19702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:07.604503 19702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:07.605547 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "5aaab44d67a34661a607608ca9158b8b"
format_stamp: "Formatted at 2026-08-07 15:11:07 on dist-test-slave-5bn2"
server_key: "a7219379d0477c8dbdcddd9a0d4a792a"
server_key_iv: "d3e3e31dede5f75005d7c93bcf141eb2"
server_key_version: "encryptionkey@0"
I20260807 15:11:07.605882 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "5aaab44d67a34661a607608ca9158b8b"
format_stamp: "Formatted at 2026-08-07 15:11:07 on dist-test-slave-5bn2"
server_key: "a7219379d0477c8dbdcddd9a0d4a792a"
server_key_iv: "d3e3e31dede5f75005d7c93bcf141eb2"
server_key_version: "encryptionkey@0"
I20260807 15:11:07.607178 19702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:07.607967 19722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:07.608109 19702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:07.608197 19702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "5aaab44d67a34661a607608ca9158b8b"
format_stamp: "Formatted at 2026-08-07 15:11:07 on dist-test-slave-5bn2"
server_key: "a7219379d0477c8dbdcddd9a0d4a792a"
server_key_iv: "d3e3e31dede5f75005d7c93bcf141eb2"
server_key_version: "encryptionkey@0"
I20260807 15:11:07.608258 19702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:07.624802 19702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:07.626611 19702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:07.626742 19702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:07.630692 19702 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:41143
I20260807 15:11:07.630733 19796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:41143 every 8 connection(s)
I20260807 15:11:07.631088 19702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:07.631907 19797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:07.633149 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19702
I20260807 15:11:07.633251 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:07.633422 18948 external_mini_cluster.cc:1428] Setting key 8d0bb953fa6d56a797e7f7b027605300
I20260807 15:11:07.634351 19797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b: Bootstrap starting.
I20260807 15:11:07.635010 19797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:07.635274 19797 log.cc:824] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:07.635949 19797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b: No bootstrap required, opened a new log
I20260807 15:11:07.638185 19797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaab44d67a34661a607608ca9158b8b" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41143 } }
I20260807 15:11:07.638337 19797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:07.638373 19797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aaab44d67a34661a607608ca9158b8b, State: Initialized, Role: FOLLOWER
I20260807 15:11:07.638494 19797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaab44d67a34661a607608ca9158b8b" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41143 } }
I20260807 15:11:07.638574 19797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:07.638607 19797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:07.638680 19797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:07.638935 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:07.633937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50141 (local address 127.18.129.62:41143)
0807 15:11:07.634089 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:07.634097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:07.634122 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:11:07.634529 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:07.634536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:07.634582 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:07.634703 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:07.634964 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:07.636008 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:07.637084 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:07.637492 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:07.637573 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:07.638244 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:07.638247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:07.638248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:07.638424 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:07.638463 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:07.638475 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:07.638523 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:07.638629 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 15:11:07.638702 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260807 15:11:07.639357 19797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaab44d67a34661a607608ca9158b8b" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41143 } }
I20260807 15:11:07.639467 19797 leader_election.cc:302] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5aaab44d67a34661a607608ca9158b8b; no voters:
I20260807 15:11:07.639619 19797 leader_election.cc:288] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:07.639640 19802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:07.639829 19802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [term 1 LEADER]: Becoming Leader. State: Replica: 5aaab44d67a34661a607608ca9158b8b, State: Running, Role: LEADER
I20260807 15:11:07.639822 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:07.639915 19797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:07.639956 19802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaab44d67a34661a607608ca9158b8b" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41143 } }
I20260807 15:11:07.640298 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5aaab44d67a34661a607608ca9158b8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaab44d67a34661a607608ca9158b8b" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41143 } } }
I20260807 15:11:07.640424 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:07.640702 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5aaab44d67a34661a607608ca9158b8b. Latest consensus state: current_term: 1 leader_uuid: "5aaab44d67a34661a607608ca9158b8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aaab44d67a34661a607608ca9158b8b" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41143 } } }
I20260807 15:11:07.640810 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:07.640966 19814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:07.641458 19814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:07.642944 19814 catalog_manager.cc:1347] Generated new cluster ID: b47597681bc845adbc11befd5860d02d
I20260807 15:11:07.642999 19814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:07.650223 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:07.641295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43863 (local address 127.18.129.62:41143)
0807 15:11:07.641328 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:07.641330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:07.647032 (+ 5702us) server_negotiation.cc:407] Connection header received
0807 15:11:07.647068 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:07.647069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:07.647080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:07.647139 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:07.647286 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:07.648040 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:07.648989 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:07.649300 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:07.649364 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:07.649987 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:07.649988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:07.649989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:07.649993 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:07.650010 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:07.650015 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:07.650070 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:07.650124 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:11:07.650146 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:11:07.671195 19814 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:07.671768 19814 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:07.676640 19814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5aaab44d67a34661a607608ca9158b8b: Generated new TSK 0
I20260807 15:11:07.676821 19814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:07.679854 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19702
2026-08-07T15:11:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:07 dist-test-slave-5bn2 krb5kdc[19833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:07 dist-test-slave-5bn2 krb5kdc[19833](info): set up 2 sockets
Aug 07 15:11:07 dist-test-slave-5bn2 krb5kdc[19833](info): commencing operation
krb5kdc: starting...
W20260807 15:11:08.821631 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:08 dist-test-slave-5bn2 krb5kdc[19833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:11:08.942200 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:41377
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:41377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:09.023187 19852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:09.023401 19852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:09.023423 19852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:09.024569 19852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:09.024613 19852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:09.024626 19852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:09.024653 19852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:09.024678 19852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:09.026144 19852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:41377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:41377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:09.026383 19852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:09.026604 19852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:09.031402 19859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:09.031517 19852 server_base.cc:1034] running on GCE node
W20260807 15:11:09.031376 19866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:09.031383 19860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:09.031901 19852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:09.032173 19852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:09.033331 19852 hybrid_clock.cc:648] HybridClock initialized: now 1786115469033317 us; error 27 us; skew 500 ppm
Aug 07 15:11:09 dist-test-slave-5bn2 krb5kdc[19833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115469, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:09.040694 19852 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:11:09.041143 19852 webserver.cc:460] Webserver started at http://127.18.129.62:33351/ using document root <none> and password file <none>
I20260807 15:11:09.041374 19852 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:09.041424 19852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:09.041551 19852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:09.042651 19852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "f1806826b3ca4b7799028c33b3fba6e8"
format_stamp: "Formatted at 2026-08-07 15:11:09 on dist-test-slave-5bn2"
server_key: "d59daa5f82f46548b8d410a2a6c58030"
server_key_iv: "42eb74aca8885b0fcb98c83810340e8b"
server_key_version: "encryptionkey@0"
I20260807 15:11:09.043027 19852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "f1806826b3ca4b7799028c33b3fba6e8"
format_stamp: "Formatted at 2026-08-07 15:11:09 on dist-test-slave-5bn2"
server_key: "d59daa5f82f46548b8d410a2a6c58030"
server_key_iv: "42eb74aca8885b0fcb98c83810340e8b"
server_key_version: "encryptionkey@0"
I20260807 15:11:09.044307 19852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:09.045096 19873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:09.045252 19852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:09.045354 19852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "f1806826b3ca4b7799028c33b3fba6e8"
format_stamp: "Formatted at 2026-08-07 15:11:09 on dist-test-slave-5bn2"
server_key: "d59daa5f82f46548b8d410a2a6c58030"
server_key_iv: "42eb74aca8885b0fcb98c83810340e8b"
server_key_version: "encryptionkey@0"
I20260807 15:11:09.045420 19852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:09.048830 19852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:09.050632 19852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:09.050765 19852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:09.054734 19852 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:41377
I20260807 15:11:09.054777 19944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:41377 every 8 connection(s)
I20260807 15:11:09.055151 19852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:09.056151 19947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:09.056836 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 19852
I20260807 15:11:09.056947 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:09.057184 18948 external_mini_cluster.cc:1428] Setting key ffb78075a8de4f6292fe3a888cefaa1a
I20260807 15:11:09.059270 19947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8: Bootstrap starting.
I20260807 15:11:09.059760 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:09.057742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43691 (local address 127.18.129.62:41377)
0807 15:11:09.057908 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:09.057916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:09.057948 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:11:09.058382 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:09.058389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:09.058421 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:09.058597 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:09.059273 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:09.059276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:09.059276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:09.059280 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:09.059325 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:09.059340 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:09.059407 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:09.059569 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:11:09.059613 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260807 15:11:09.059963 19947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:09.060231 19947 log.cc:824] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:09.060699 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:09.061007 19947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8: No bootstrap required, opened a new log
I20260807 15:11:09.063189 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:09.062108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36215 (local address 127.18.129.62:41377)
0807 15:11:09.062142 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:09.062144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:09.062251 (+ 107us) server_negotiation.cc:407] Connection header received
0807 15:11:09.062312 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:09.062313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:09.062318 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:09.062353 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:09.062936 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:09.062937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:09.062937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:09.062942 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:09.062957 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:09.062961 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:09.063018 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:09.063073 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 15:11:09.063085 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:11:09.063107 19947 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1806826b3ca4b7799028c33b3fba6e8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41377 } }
I20260807 15:11:09.063309 19947 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:09.063371 19947 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1806826b3ca4b7799028c33b3fba6e8, State: Initialized, Role: FOLLOWER
I20260807 15:11:09.063496 19947 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1806826b3ca4b7799028c33b3fba6e8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41377 } }
I20260807 15:11:09.063582 19947 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:09.063623 19947 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:09.063673 19947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:09.064232 19947 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1806826b3ca4b7799028c33b3fba6e8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41377 } }
I20260807 15:11:09.064330 19947 leader_election.cc:302] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1806826b3ca4b7799028c33b3fba6e8; no voters:
I20260807 15:11:09.064494 19947 leader_election.cc:288] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:09.064538 19964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:09.064689 19964 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [term 1 LEADER]: Becoming Leader. State: Replica: f1806826b3ca4b7799028c33b3fba6e8, State: Running, Role: LEADER
I20260807 15:11:09.064733 19947 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:09.064811 19964 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1806826b3ca4b7799028c33b3fba6e8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41377 } }
I20260807 15:11:09.065191 19965 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1806826b3ca4b7799028c33b3fba6e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1806826b3ca4b7799028c33b3fba6e8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41377 } } }
I20260807 15:11:09.065284 19965 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:09.065371 19966 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1806826b3ca4b7799028c33b3fba6e8. Latest consensus state: current_term: 1 leader_uuid: "f1806826b3ca4b7799028c33b3fba6e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1806826b3ca4b7799028c33b3fba6e8" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 41377 } } }
I20260807 15:11:09.065582 19966 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:09.065614 19977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:09.066102 19977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:09.067649 19977 catalog_manager.cc:1347] Generated new cluster ID: 5549bb85972e4897b2696630fd138fa2
I20260807 15:11:09.067701 19977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:09.074386 19977 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:09.074595 19977 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:09.079276 19977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1806826b3ca4b7799028c33b3fba6e8: Generated new TSK 0
I20260807 15:11:09.079455 19977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:09.084312 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 19852
2026-08-07T15:11:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1409 ms)
[----------] 18 tests from AuthTokenIssuingTest (25632 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:09 dist-test-slave-5bn2 krb5kdc[19993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:09 dist-test-slave-5bn2 krb5kdc[19993](info): set up 2 sockets
Aug 07 15:11:09 dist-test-slave-5bn2 krb5kdc[19993](info): commencing operation
krb5kdc: starting...
W20260807 15:11:10.259184 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:11:10.385746 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46367
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46367,127.18.129.61:38645,127.18.129.60:43641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:10.467082 20012 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:11:10.467307 20012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:10.467332 20012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:10.467360 20012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:10.468499 20012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:10.468540 20012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:10.468559 20012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:10.468575 20012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:10.468590 20012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:10.470077 20012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46367,127.18.129.61:38645,127.18.129.60:43641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:46367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:10.470346 20012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:10.470554 20012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:10.475461 20022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:10.475457 20024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:10.475539 20012 server_base.cc:1034] running on GCE node
W20260807 15:11:10.475457 20021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:10.475924 20012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:10.476215 20012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:10.477360 20012 hybrid_clock.cc:648] HybridClock initialized: now 1786115470477338 us; error 31 us; skew 500 ppm
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:10.479900 20012 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:11:10.480342 20012 webserver.cc:460] Webserver started at http://127.18.129.62:46245/ using document root <none> and password file <none>
I20260807 15:11:10.480562 20012 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:10.480636 20012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:10.480762 20012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:10.481837 20012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "27f056e6dbbf4aa596e91f2a92162dde"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "1579bff34821f9f4042fc02da8505148"
server_key_iv: "55112205ce4633653b6b947355924f31"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.482191 20012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "27f056e6dbbf4aa596e91f2a92162dde"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "1579bff34821f9f4042fc02da8505148"
server_key_iv: "55112205ce4633653b6b947355924f31"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.483546 20012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:10.484239 20033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:10.484406 20012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:10.484498 20012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "27f056e6dbbf4aa596e91f2a92162dde"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "1579bff34821f9f4042fc02da8505148"
server_key_iv: "55112205ce4633653b6b947355924f31"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.484558 20012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:10.496263 20012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:10.498071 20012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:10.498210 20012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:10.502252 20012 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:46367
I20260807 15:11:10.502290 20106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:46367 every 8 connection(s)
I20260807 15:11:10.502637 20012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:10.503654 20107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:10.505179 20107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:10.510566 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20012
I20260807 15:11:10.510677 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:10.510918 18948 external_mini_cluster.cc:1428] Setting key 3f5395d9620bd3de2e05ea07827a7b62
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:11:10.514914 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.505801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46367 (local address 127.0.0.1:36966)
0807 15:11:10.505993 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.506035 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.506497 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.506767 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.506770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.507085 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.507239 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.507243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.508280 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.508281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.508945 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.508947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.509407 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.509409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.509473 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.509957 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.509965 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.511639 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.513416 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.513423 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.513425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.513652 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.513908 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.513909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.513909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.513986 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.514386 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.514389 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.514443 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.514506 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.514645 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260807 15:11:10.514921 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.505854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.18.129.62:46367)
0807 15:11:10.506003 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.506028 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.506127 (+ 99us) server_negotiation.cc:407] Connection header received
0807 15:11:10.506666 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.506668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.506684 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.506757 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.507331 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.508192 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.509026 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.509320 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.509421 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.511726 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.511733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.511734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.511741 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.513333 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.513734 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.513737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.513738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.513842 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.514048 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.514048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.514049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.514320 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.514399 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.514560 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.514724 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260807 15:11:10.515869 20107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:10.516203 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.516098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34440)
0807 15:11:10.516138 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.516180 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 15:11:10.516341 20034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
I20260807 15:11:10.516767 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.516683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34456)
0807 15:11:10.516724 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.516749 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:11:10.516870 20107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:38645: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.129.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab.
Entry for principal kudu/127.18.129.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab.
I20260807 15:11:10.535837 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.61:38645
--webserver_interface=127.18.129.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab
--principal=kudu/127.18.129.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46367,127.18.129.61:38645,127.18.129.60:43641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:11:10.566123 20107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } attempt: 1
I20260807 15:11:10.566699 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.566538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34468)
0807 15:11:10.566613 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.566663 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:11:10.567062 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.566985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34476)
0807 15:11:10.567019 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.567044 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 15:11:10.567149 20107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:38645: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
W20260807 15:11:10.618949 20117 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:11:10.619161 20117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:10.619176 20117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:10.619194 20117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:10.620283 20117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:10.620317 20117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:10.620329 20117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:10.620340 20117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:10.620350 20117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:10.621838 20117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46367,127.18.129.61:38645,127.18.129.60:43641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab
--principal=kudu/127.18.129.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.61:38645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.129.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:10.622032 20117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:10.622206 20117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:10.626945 20130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:10.626940 20124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:10.627038 20125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:10.627090 20117 server_base.cc:1034] running on GCE node
I20260807 15:11:10.627430 20117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:10.627712 20117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:10.628854 20117 hybrid_clock.cc:648] HybridClock initialized: now 1786115470628842 us; error 27 us; skew 500 ppm
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:10.631345 20117 init.cc:377] Logged in from keytab as kudu/127.18.129.61@KRBTEST.COM (short username kudu)
I20260807 15:11:10.631771 20117 webserver.cc:460] Webserver started at http://127.18.129.61:42723/ using document root <none> and password file <none>
I20260807 15:11:10.631969 20117 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:10.632015 20117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:10.632135 20117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:10.633183 20117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/data/instance:
uuid: "8aeb1497bae64361accc141550470cf9"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "fad9ae6901542d7ab13843ac45bf2702"
server_key_iv: "e224796b6e2ea7e1f19dc285d2b83c1f"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.633524 20117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/wal/instance:
uuid: "8aeb1497bae64361accc141550470cf9"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "fad9ae6901542d7ab13843ac45bf2702"
server_key_iv: "e224796b6e2ea7e1f19dc285d2b83c1f"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.634799 20117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:11:10.635341 20107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } attempt: 2
I20260807 15:11:10.635664 20140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:10.635805 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.635668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34490)
0807 15:11:10.635727 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.635767 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:11:10.635854 20117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:10.635936 20117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/wal
uuid: "8aeb1497bae64361accc141550470cf9"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "fad9ae6901542d7ab13843ac45bf2702"
server_key_iv: "e224796b6e2ea7e1f19dc285d2b83c1f"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.636008 20117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:10.636178 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.636087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34504)
0807 15:11:10.636130 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.636153 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:11:10.636289 20107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:38645: Network error: Client connection negotiation failed: client connection to 127.18.129.61:38645: connect: Connection refused (error 111)
I20260807 15:11:10.652403 20117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:10.654618 20117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:10.654734 20117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:10.658541 20117 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.61:38645
I20260807 15:11:10.658591 20213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.61:38645 every 8 connection(s)
I20260807 15:11:10.658951 20117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
I20260807 15:11:10.660022 20214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:10.661166 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20117
I20260807 15:11:10.661253 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-1/wal/instance
I20260807 15:11:10.661422 18948 external_mini_cluster.cc:1428] Setting key d0f384432b7e07509b1269866f950d28
I20260807 15:11:10.661689 20214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.129.60@KRBTEST.COM; defaulting to no policy
I20260807 15:11:10.671428 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.662391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.18.129.62:46367)
0807 15:11:10.662453 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.662455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.662667 (+ 212us) server_negotiation.cc:407] Connection header received
0807 15:11:10.663247 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.663249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.663263 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.663307 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.664121 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.664707 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.665696 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.665995 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.666069 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.668483 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.668485 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.668485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.668494 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.670131 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.670584 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.670585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.670586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.670645 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.670839 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.670840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.670840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.670974 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.671025 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.671280 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.671354 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:11:10.671679 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.662322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46367 (local address 127.0.0.1:36968)
0807 15:11:10.662563 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.662593 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.663109 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.663400 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.663405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.663747 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.664038 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.664044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.664804 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.664804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.665606 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.665609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.666084 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.666085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.666211 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.666734 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.666742 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.668348 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.670236 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.670244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.670246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.670510 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.670701 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.670701 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.670702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.670801 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.671024 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.671029 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.671157 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.671259 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.671423 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":89,"threads_started":1}
I20260807 15:11:10.672304 20214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.60@KRBTEST.COM" created.
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:11:10.679730 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.672575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34514)
0807 15:11:10.672623 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.672635 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.672677 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.672993 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.672994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.673148 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.673210 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.673212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.674062 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.674063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.674564 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.674565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.675188 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.675189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.675266 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.675670 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.675674 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.676924 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.678561 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.678562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.678563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.678775 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.679001 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.679002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.679002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.679070 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.679349 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.679349 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.679397 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.679474 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.679580 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 15:11:10.679739 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.672664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.18.129.61:38645)
0807 15:11:10.672861 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.672865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.672881 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:11:10.672899 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.672901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.672911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.672974 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.673273 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.673967 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.674630 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.675098 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.675165 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.677035 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.677044 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.677044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.677050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.678475 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.678883 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.678886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.678887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.678942 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.679151 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.679151 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.679152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.679291 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.679350 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.679499 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.679627 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":171,"thread_start_us":107,"threads_started":1}
I20260807 15:11:10.680449 20214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:10.680872 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.680738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41066)
0807 15:11:10.680789 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.680845 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:11:10.681030 20143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
I20260807 15:11:10.681443 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.681345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41082)
0807 15:11:10.681390 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.681420 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:11:10.681561 20214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:43641: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab.
Entry for principal kudu/127.18.129.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab.
I20260807 15:11:10.689528 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.60:43641
--webserver_interface=127.18.129.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab
--principal=kudu/127.18.129.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:36677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:46367,127.18.129.61:38645,127.18.129.60:43641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:11:10.727463 20107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } attempt: 3
I20260807 15:11:10.730813 20214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } attempt: 1
I20260807 15:11:10.731287 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.731143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41086)
0807 15:11:10.731209 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.731244 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 15:11:10.731680 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.731572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41098)
0807 15:11:10.731621 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.731654 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 15:11:10.731783 20214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:43641: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:11:10.735517 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.727913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34526)
0807 15:11:10.728011 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.728026 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.728097 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.728235 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.728236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.728474 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.728550 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.728553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.729528 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.729529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.730113 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.730115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.730628 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.730629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.730721 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.731283 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.731288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.732643 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.734486 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.734487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.734488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.734661 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.734911 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.734912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.734913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.734985 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.735255 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.735255 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.735298 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.735343 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.735424 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 15:11:10.735534 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.728009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.18.129.61:38645)
0807 15:11:10.728094 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.728097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.728110 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:11:10.728146 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.728147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.728159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.728221 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.728633 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.729419 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.730196 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.730533 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.730606 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.732745 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.732747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.732747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.732755 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.734387 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.734734 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.734735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.734735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.734799 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.735051 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.735052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.735052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.735197 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.735266 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.735370 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.735451 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:11:10.735994 20107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:10.736311 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.736225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41104)
0807 15:11:10.736265 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.736291 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 15:11:10.736640 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.736583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41110)
0807 15:11:10.736606 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.736627 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260807 15:11:10.736725 20107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:43641: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
I20260807 15:11:10.767885 20107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } attempt: 1
I20260807 15:11:10.768380 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.768252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41126)
0807 15:11:10.768311 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.768345 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:11:10.768762 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.768677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41138)
0807 15:11:10.768718 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.768745 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 15:11:10.768854 20107 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:43641: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
W20260807 15:11:10.775342 20229 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:11:10.775576 20229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:10.775611 20229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:10.775632 20229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:10.776777 20229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:10.776808 20229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:10.776819 20229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:10.776839 20229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:10.776850 20229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:10.778317 20229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:36677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:46367,127.18.129.61:38645,127.18.129.60:43641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab
--principal=kudu/127.18.129.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.60:43641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.129.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:10.778527 20229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:10.778718 20229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:10.783656 20237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:10.783663 20238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:10.783723 20229 server_base.cc:1034] running on GCE node
W20260807 15:11:10.783676 20240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:10.784101 20229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:10.784380 20229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:10.785522 20229 hybrid_clock.cc:648] HybridClock initialized: now 1786115470785506 us; error 35 us; skew 500 ppm
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:10.788072 20229 init.cc:377] Logged in from keytab as kudu/127.18.129.60@KRBTEST.COM (short username kudu)
I20260807 15:11:10.788548 20229 webserver.cc:460] Webserver started at http://127.18.129.60:44695/ using document root <none> and password file <none>
I20260807 15:11:10.788759 20229 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:10.788807 20229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:10.788937 20229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:10.790023 20229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/data/instance:
uuid: "a8d5a8bb698544bb8164eb0a762a5c76"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "6cbd3245c7ea7985e8303ca010a2f25f"
server_key_iv: "a9d21b935f6a3b234144891eddc7fd19"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.790397 20229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/wal/instance:
uuid: "a8d5a8bb698544bb8164eb0a762a5c76"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "6cbd3245c7ea7985e8303ca010a2f25f"
server_key_iv: "a9d21b935f6a3b234144891eddc7fd19"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.791761 20229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:10.792438 20256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:10.792591 20229 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:10.792680 20229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/wal
uuid: "a8d5a8bb698544bb8164eb0a762a5c76"
format_stamp: "Formatted at 2026-08-07 15:11:10 on dist-test-slave-5bn2"
server_key: "6cbd3245c7ea7985e8303ca010a2f25f"
server_key_iv: "a9d21b935f6a3b234144891eddc7fd19"
server_key_version: "encryptionkey@0"
I20260807 15:11:10.792749 20229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:10.799918 20214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } attempt: 2
I20260807 15:11:10.800340 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.800220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41140)
0807 15:11:10.800274 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.800309 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:11:10.800663 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.800577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41142)
0807 15:11:10.800615 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.800639 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 15:11:10.800740 20214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:43641: Network error: Client connection negotiation failed: client connection to 127.18.129.60:43641: connect: Connection refused (error 111)
I20260807 15:11:10.829648 20229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:10.832454 20229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:10.832633 20229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:10.836840 20229 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.60:43641
I20260807 15:11:10.836894 20327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.60:43641 every 8 connection(s)
I20260807 15:11:10.837241 20229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
I20260807 15:11:10.838325 20328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:10.840111 20328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:10.844033 20107 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } attempt: 2
I20260807 15:11:10.845520 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20229
I20260807 15:11:10.845639 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115197351496-18948-0/minicluster-data/master-2/wal/instance
I20260807 15:11:10.845845 18948 external_mini_cluster.cc:1428] Setting key 4697186fedc053afc21a168a3a88d875
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:11:10.850543 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.840865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36970 (local address 127.18.129.62:46367)
0807 15:11:10.840923 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.840925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.841197 (+ 272us) server_negotiation.cc:407] Connection header received
0807 15:11:10.841670 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.841671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.841684 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.841756 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.842422 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.842997 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.844069 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.844303 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.844379 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.847188 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.847190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.847190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.847199 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.848893 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.849435 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.849436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.849436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.849501 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.849782 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.849783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.849784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.849918 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.849989 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.850351 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.850447 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:11:10.850685 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.840828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:46367 (local address 127.0.0.1:36970)
0807 15:11:10.841022 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.841069 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.841554 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.841825 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.841829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.842136 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.842347 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.842351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.843086 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.843088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.843958 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.843961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.844419 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.844421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.844584 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.845106 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.845115 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.847096 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.848992 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.849001 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.849003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.849360 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.849572 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.849572 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.849573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.849720 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.849996 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.850005 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.850196 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.850296 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.850462 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:11:10.851424 20328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:10.852732 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.844349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41146)
0807 15:11:10.844411 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.844422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.844458 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.844791 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.844792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.844946 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.844989 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.844991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.845933 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.845935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.846611 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.846614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.847233 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.847234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.847290 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.847699 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.847703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.849060 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.851566 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.851567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.851567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.851787 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.851978 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.851979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.851979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.852044 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.852391 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.852392 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.852439 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.852510 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.852621 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:11:10.852744 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.844452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.18.129.60:43641)
0807 15:11:10.844632 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.844636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.844652 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:11:10.844676 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.844678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.844694 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.844756 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.845064 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.845831 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.846685 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.847167 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.847219 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.849161 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.849167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.849167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.849174 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.851479 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.851858 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.851861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.851861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.851921 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.852112 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.852113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.852113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.852320 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.852395 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.852554 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.852627 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260807 15:11:10.854770 20107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde: Bootstrap starting.
I20260807 15:11:10.855633 20107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:10.855923 20107 log.cc:824] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:11:10.856355 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.846519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.18.129.60:43641)
0807 15:11:10.846651 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.846653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.846660 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:10.846676 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.846677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.846684 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.846744 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.847459 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.848051 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.849038 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.849294 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.849349 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.851889 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.851891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.851891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.851898 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.854556 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.854966 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.854968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.854968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.855031 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.855230 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.855231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.855231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.855337 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.855476 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.856132 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.856230 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260807 15:11:10.856777 20107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde: No bootstrap required, opened a new log
I20260807 15:11:10.859225 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.851760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34534 (local address 127.18.129.61:38645)
0807 15:11:10.851806 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.851808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.851816 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:10.851840 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.851841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.851852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.851906 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.852254 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.852799 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.853680 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.854211 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.854288 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.856404 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.856406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.856407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.856416 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.857908 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.858340 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.858341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.858341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.858414 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.858623 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.858624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.858624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.858765 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.858832 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.859056 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.859142 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:11:10.859145 20107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } }
I20260807 15:11:10.859292 20107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f056e6dbbf4aa596e91f2a92162dde, State: Initialized, Role: FOLLOWER
I20260807 15:11:10.859381 20107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } }
I20260807 15:11:10.859545 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.851679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:38645 (local address 127.0.0.1:34534)
0807 15:11:10.851721 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.851730 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.851763 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.851911 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.851912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.852071 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.852141 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.852142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.852872 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.852873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.853586 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.853589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.854297 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.854298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.854370 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.855021 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.855025 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.856311 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.858001 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.858002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.858002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.858258 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.858488 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.858489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.858489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.858552 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.858888 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.858889 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.858935 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.859353 (+ 418us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.859461 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:11:10.859827 20339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.859877 20107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:10.859963 20339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:10.860132 20328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } has no permanent_uuid. Determining permanent_uuid...
W20260807 15:11:10.861927 20353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:11:10.862048 20353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:11:10.865433 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.857430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36976 (local address 127.18.129.62:46367)
0807 15:11:10.858018 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.858020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.858028 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:10.858046 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.858047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.858059 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.858110 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.858665 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.859181 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.860099 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.860342 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.860400 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.862661 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.862662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.862662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.862670 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.863938 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.864419 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.864420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.864420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.864477 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.864638 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.864639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.864639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.864756 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.864908 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.865276 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.865351 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564}
I20260807 15:11:10.863039 20012 master_runner.cc:186] Error getting master registration for 127.18.129.61:38645: OK, instance_id { permanent_uuid: "8aeb1497bae64361accc141550470cf9" instance_seqno: 1786115470657501 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:11:10.866163 20012 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:11:10.868340 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.860383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41162)
0807 15:11:10.861442 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.861453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.861550 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.861708 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.861709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.861849 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.861885 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.861887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.862692 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.862693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.863372 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.863375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.863846 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.863847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.863889 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.864190 (+ 301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.864195 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.865668 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.867242 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.867243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.867243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.867435 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.867651 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.867651 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.867652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.867711 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.868030 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.868031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.868077 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.868144 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.868243 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1042}
I20260807 15:11:10.868434 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.860413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.18.129.60:43641)
0807 15:11:10.860445 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.860447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.861501 (+ 1054us) server_negotiation.cc:407] Connection header received
0807 15:11:10.861625 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.861626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.861637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.861660 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.861977 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.862601 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.863456 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.863743 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.863782 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.865734 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.865735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.865735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.865742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.867173 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.867514 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.867516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.867516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.867583 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.867771 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.867772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.867773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.867966 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.868039 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.868179 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.868326 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":15}
I20260807 15:11:10.869954 20328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76: Bootstrap starting.
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:11:10.870755 20328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:10.871151 20328 log.cc:824] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:10.872150 20328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76: No bootstrap required, opened a new log
I20260807 15:11:10.873536 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.866507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.18.129.61:38645)
0807 15:11:10.866551 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.866552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.866562 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:10.866590 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.866591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.866600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.866646 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.867278 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.867896 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.868768 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.868982 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.869035 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.871051 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.871053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.871053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.871061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.872542 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.872835 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.872836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.872836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.872890 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.873046 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.873047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.873048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.873190 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.873346 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.873381 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.873450 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:11:10.874234 20328 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } }
I20260807 15:11:10.874411 20328 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d5a8bb698544bb8164eb0a762a5c76, State: Initialized, Role: FOLLOWER
I20260807 15:11:10.874552 20328 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } }
I20260807 15:11:10.874886 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:10.874956 20328 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:10.874913 20359 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.875020 20359 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:10.875819 20069 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "27f056e6dbbf4aa596e91f2a92162dde"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36976
I20260807 15:11:10.875941 20069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:11:10.875995 20069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:10.876998 20069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } }
I20260807 15:11:10.877254 20069 leader_election.cc:288] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [CANDIDATE]: Term 1 election: Requested vote from peers 8aeb1497bae64361accc141550470cf9 (127.18.129.61:38645), a8d5a8bb698544bb8164eb0a762a5c76 (127.18.129.60:43641)
W20260807 15:11:10.877082 20371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 15:11:10.877704 20292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "27f056e6dbbf4aa596e91f2a92162dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a8d5a8bb698544bb8164eb0a762a5c76"
I20260807 15:11:10.880228 20292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:10.881896 20292 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27f056e6dbbf4aa596e91f2a92162dde in term 1.
W20260807 15:11:10.882043 20371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:10.882153 20036 leader_election.cc:302] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27f056e6dbbf4aa596e91f2a92162dde, a8d5a8bb698544bb8164eb0a762a5c76; no voters:
I20260807 15:11:10.882350 20339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:10.882519 20339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [term 1 LEADER]: Becoming Leader. State: Replica: 27f056e6dbbf4aa596e91f2a92162dde, State: Running, Role: LEADER
I20260807 15:11:10.882627 20339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } }
I20260807 15:11:10.883085 20339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27f056e6dbbf4aa596e91f2a92162dde. Latest consensus state: current_term: 1 leader_uuid: "27f056e6dbbf4aa596e91f2a92162dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.883167 20339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:10.881771 20229 master_runner.cc:186] Error getting master registration for 127.18.129.61:38645: OK, instance_id { permanent_uuid: "8aeb1497bae64361accc141550470cf9" instance_seqno: 1786115470657501 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:11:10.885454 20229 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:11:10.885779 20382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:10.886003 20382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:10.887813 20292 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [term 1 FOLLOWER]: Refusing update from remote peer 27f056e6dbbf4aa596e91f2a92162dde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:10.888233 20339 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:10.889549 20359 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27f056e6dbbf4aa596e91f2a92162dde. Latest consensus state: current_term: 1 leader_uuid: "27f056e6dbbf4aa596e91f2a92162dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.889654 20359 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:10.891855 20214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } attempt: 3
Aug 07 15:11:10 dist-test-slave-5bn2 krb5kdc[19993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:11:10.901687 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.881505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.18.129.62:46367)
0807 15:11:10.881546 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.881548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.891558 (+ 10010us) server_negotiation.cc:407] Connection header received
0807 15:11:10.891660 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.891661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.891673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.891726 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.893582 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.894365 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.896138 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.896394 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.896431 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.898473 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.898474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.898475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.898494 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.900330 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.900760 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.900761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.900762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.900825 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.901019 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.901020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.901020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.901160 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.901262 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.901488 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.901577 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:11:10.904527 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.878649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34554 (local address 127.18.129.61:38645)
0807 15:11:10.878689 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.878691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.891582 (+ 12891us) server_negotiation.cc:407] Connection header received
0807 15:11:10.891703 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.891704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.891726 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.891756 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.892916 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.893370 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.898188 (+ 4818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.898510 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.898564 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.900934 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.900949 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.900950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.900957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.902639 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.903434 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.903435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.903436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.903520 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.903836 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.903837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.903837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.903982 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.904076 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.904386 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.904443 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:11:10.904650 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.892232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41180 (local address 127.18.129.60:43641)
0807 15:11:10.892279 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.892281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.892290 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:10.892311 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.892312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.892323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.892382 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.892696 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.893450 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.894451 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.894875 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.894946 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.900694 (+ 5748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.900696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.900696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.900704 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.903395 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.903755 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.903756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.903756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.903814 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.903984 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.903985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.903985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.904121 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.904179 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.904495 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.904569 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:11:10.904804 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.892137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:43641 (local address 127.0.0.1:41180)
0807 15:11:10.892183 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:10.892193 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:10.892229 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:10.892387 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:10.892387 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:10.892561 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:10.892621 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.892623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.893531 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.893532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.894230 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:10.894233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.897735 (+ 3502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.897736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:10.897812 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.898644 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:10.898648 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:10.900586 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:10.903469 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.903470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.903471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.903681 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.903875 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:10.903876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:10.903876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:10.903924 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:10.904204 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:10.904205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:10.904247 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 15:11:10.904303 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 15:11:10.904404 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:11:10.906524 20214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9: Bootstrap starting.
I20260807 15:11:10.907325 20214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:10.907645 20214 log.cc:824] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:10.908643 20214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9: No bootstrap required, opened a new log
I20260807 15:11:10.910715 20214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } }
I20260807 15:11:10.910960 20214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aeb1497bae64361accc141550470cf9, State: Initialized, Role: FOLLOWER
I20260807 15:11:10.911062 20214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } }
I20260807 15:11:10.913097 20391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.913221 20177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:10.913396 20391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:10.914254 20177 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [term 1 FOLLOWER]: Refusing update from remote peer 27f056e6dbbf4aa596e91f2a92162dde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:10.914374 20339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27f056e6dbbf4aa596e91f2a92162dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.914476 20339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:10.913221 20178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "27f056e6dbbf4aa596e91f2a92162dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8aeb1497bae64361accc141550470cf9"
I20260807 15:11:10.915067 20386 mvcc.cc:205] Tried to move back new op lower bound from 7315928968753618944 to 7315928968736075776. Current Snapshot: MvccSnapshot[applied={T|T < 7315928968753618944}]
I20260807 15:11:10.915125 20380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:10.915329 20178 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27f056e6dbbf4aa596e91f2a92162dde in term 1.
I20260807 15:11:10.915442 20359 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27f056e6dbbf4aa596e91f2a92162dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.915522 20359 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:10.915795 20384 mvcc.cc:205] Tried to move back new op lower bound from 7315928968753618944 to 7315928968736075776. Current Snapshot: MvccSnapshot[applied={T|T < 7315928968753618944}]
I20260807 15:11:10.913146 20214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:10.916567 20391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27f056e6dbbf4aa596e91f2a92162dde. Latest consensus state: current_term: 1 leader_uuid: "27f056e6dbbf4aa596e91f2a92162dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.917234 20391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:10.917296 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:10.891149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.18.129.60:43641)
0807 15:11:10.891212 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:10.891214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:10.896472 (+ 5258us) server_negotiation.cc:407] Connection header received
0807 15:11:10.896493 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:10.896494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:10.896504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:10.896550 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:10.902124 (+ 5574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.902877 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.910947 (+ 8070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:10.911340 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:10.911396 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:10.913658 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:10.913660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:10.913660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:10.913669 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:10.915826 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.916359 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.916361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.916361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.916425 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:10.916654 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:10.916655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:10.916655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:10.916794 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:10.916886 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:10.917118 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 15:11:10.917209 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:11:10.917846 20395 mvcc.cc:205] Tried to move back new op lower bound from 7315928968753618944 to 7315928968736075776. Current Snapshot: MvccSnapshot[applied={T|T < 7315928968753618944}]
I20260807 15:11:10.918543 20382 catalog_manager.cc:1347] Generated new cluster ID: cde7b2d9c95847b089836c2c8f4a2e18
I20260807 15:11:10.918596 20382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:10.917304 20391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "27f056e6dbbf4aa596e91f2a92162dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.918994 20391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:10.919853 20418 catalog_manager.cc:1259] Loaded cluster ID: cde7b2d9c95847b089836c2c8f4a2e18
I20260807 15:11:10.919893 20418 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9: loading cluster ID for follower catalog manager: success
W20260807 15:11:10.919982 20418 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260807 15:11:10.920017 20418 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:10.968506 20393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "27f056e6dbbf4aa596e91f2a92162dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27f056e6dbbf4aa596e91f2a92162dde" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 46367 } } peers { permanent_uuid: "8aeb1497bae64361accc141550470cf9" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 38645 } } peers { permanent_uuid: "a8d5a8bb698544bb8164eb0a762a5c76" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 43641 } } }
I20260807 15:11:10.968658 20393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:10.970825 20382 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:10.972741 20382 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:10.980945 20382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27f056e6dbbf4aa596e91f2a92162dde: Generated new TSK 0
I20260807 15:11:10.981158 20382 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:11:11.015445 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.013002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34570 (local address 127.18.129.61:38645)
0807 15:11:11.013050 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.013052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.014024 (+ 972us) server_negotiation.cc:407] Connection header received
0807 15:11:11.014474 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.014475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.014489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.014523 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.015364 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34570: BlockingRecv error: recv got EOF from 127.0.0.1:34570 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 15:11:11.016139 20111 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.013465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.18.129.62:46367)
0807 15:11:11.013508 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.013510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.014334 (+ 824us) server_negotiation.cc:407] Connection header received
0807 15:11:11.015627 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.015628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.015641 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.015671 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.016052 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36982: BlockingRecv error: recv got EOF from 127.0.0.1:36982 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 15:11:11.016549 20337 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.013263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41196 (local address 127.18.129.60:43641)
0807 15:11:11.013319 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.013321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.014149 (+ 828us) server_negotiation.cc:407] Connection header received
0807 15:11:11.014601 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.014602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.014614 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.014644 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.016486 (+ 1842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41196: BlockingRecv error: recv got EOF from 127.0.0.1:41196 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 15:11:11.019068 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.018284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.18.129.61:38645)
0807 15:11:11.018325 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.018327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.018501 (+ 174us) server_negotiation.cc:407] Connection header received
0807 15:11:11.018608 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.018609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.018627 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.018654 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.019003 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34582: BlockingRecv error: recv got EOF from 127.0.0.1:34582 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:11:11.022131 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.021367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.18.129.61:38645)
0807 15:11:11.021406 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.021408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.021604 (+ 196us) server_negotiation.cc:407] Connection header received
0807 15:11:11.021699 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.021700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.021710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.021745 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.022068 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34588: BlockingRecv error: recv got EOF from 127.0.0.1:34588 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:11:11.026067 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.025251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34600 (local address 127.18.129.61:38645)
0807 15:11:11.025286 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.025288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.025574 (+ 286us) server_negotiation.cc:407] Connection header received
0807 15:11:11.025670 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.025671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.025681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.025708 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.026021 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34600: BlockingRecv error: recv got EOF from 127.0.0.1:34600 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 15:11:11.031991 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.031217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.18.129.61:38645)
0807 15:11:11.031250 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.031252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.031514 (+ 262us) server_negotiation.cc:407] Connection header received
0807 15:11:11.031604 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.031605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.031615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.031641 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.031943 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34612: BlockingRecv error: recv got EOF from 127.0.0.1:34612 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 15:11:11.042065 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.041227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34628 (local address 127.18.129.61:38645)
0807 15:11:11.041267 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.041269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.041511 (+ 242us) server_negotiation.cc:407] Connection header received
0807 15:11:11.041605 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.041606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.041617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.041647 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.042011 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34628: BlockingRecv error: recv got EOF from 127.0.0.1:34628 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 15:11:11.060598 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.059513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34636 (local address 127.18.129.61:38645)
0807 15:11:11.059558 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.059560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.059913 (+ 353us) server_negotiation.cc:407] Connection header received
0807 15:11:11.060052 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.060053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.060068 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.060100 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.060533 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34636: BlockingRecv error: recv got EOF from 127.0.0.1:34636 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:11:11.094625 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.093913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.18.129.61:38645)
0807 15:11:11.093964 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.093966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.094098 (+ 132us) server_negotiation.cc:407] Connection header received
0807 15:11:11.094151 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.094152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.094165 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.094214 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.094558 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34648: BlockingRecv error: recv got EOF from 127.0.0.1:34648 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 15:11:11.160499 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.159744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.18.129.61:38645)
0807 15:11:11.159804 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.159806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.159913 (+ 107us) server_negotiation.cc:407] Connection header received
0807 15:11:11.159980 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.159981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.159993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.160041 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.160436 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34658: BlockingRecv error: recv got EOF from 127.0.0.1:34658 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260807 15:11:11.290232 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.289525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.18.129.61:38645)
0807 15:11:11.289596 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.289598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.289659 (+ 61us) server_negotiation.cc:407] Connection header received
0807 15:11:11.289744 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.289745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.289757 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.289799 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.290165 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34660: BlockingRecv error: recv got EOF from 127.0.0.1:34660 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 15:11:11.547998 20224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:11.547400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.18.129.61:38645)
0807 15:11:11.547481 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:11.547484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:11.547495 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:11.547546 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:11.547547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:11.547557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:11.547598 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:11.547932 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34676: BlockingRecv error: recv got EOF from 127.0.0.1:34676 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:11:11.882642 20371 catalog_manager.cc:1259] Loaded cluster ID: cde7b2d9c95847b089836c2c8f4a2e18
I20260807 15:11:11.882727 20371 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76: loading cluster ID for follower catalog manager: success
I20260807 15:11:11.883705 20371 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76: acquiring CA information for follower catalog manager: success
I20260807 15:11:11.884088 20371 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a8d5a8bb698544bb8164eb0a762a5c76: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:11:11.921155 20418 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9: acquiring CA information for follower catalog manager: success
I20260807 15:11:11.921481 20418 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8aeb1497bae64361accc141550470cf9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:11:12.063686 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:12.061097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.18.129.61:38645)
0807 15:11:12.061285 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:12.061287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:12.061297 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:12.061328 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:12.061329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:12.061393 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:12.061462 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:12.061612 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.062125 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.062919 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.063131 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.063188 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:12.063343 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:12.063457 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:12.063526 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:12.063568 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 15:11:12.063590 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":62,"threads_started":1}
I20260807 15:11:12.067850 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:12.065500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.18.129.60:43641)
0807 15:11:12.065634 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:12.065636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:12.065643 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:12.065686 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:12.065686 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:12.065726 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:12.065794 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:12.065896 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.066390 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.067125 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.067333 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.067389 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:12.067521 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:12.067625 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:12.067691 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:12.067739 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 15:11:12.067760 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":45,"threads_started":1}
I20260807 15:11:12.072180 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:12.069245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36984 (local address 127.18.129.62:46367)
0807 15:11:12.069476 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:12.069478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:12.069555 (+ 77us) server_negotiation.cc:407] Connection header received
0807 15:11:12.069605 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:12.069606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:12.069656 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:12.069740 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:12.069908 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.070432 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.071265 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.071488 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.071545 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:12.071726 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:12.071830 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:12.071891 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:12.072062 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 15:11:12.072089 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":52,"threads_started":1}
I20260807 15:11:12.072249 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:12.069255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.18.129.61:38645)
0807 15:11:12.069293 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:12.069295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:12.069446 (+ 151us) server_negotiation.cc:407] Connection header received
0807 15:11:12.069551 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:12.069552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:12.069564 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:12.069610 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:12.069712 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.070445 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.071598 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.071812 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.071865 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:12.071984 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:12.072021 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:12.072063 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:12.072163 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 15:11:12.072187 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:11:12.072718 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:12.069286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41212 (local address 127.18.129.60:43641)
0807 15:11:12.069326 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:12.069328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:12.069583 (+ 255us) server_negotiation.cc:407] Connection header received
0807 15:11:12.069678 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:12.069679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:12.069688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:12.069726 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:12.069887 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.070576 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.071590 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:12.071843 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:12.071881 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:12.072069 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:12.072100 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:12.072150 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:12.072628 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 15:11:12.072652 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:11:12.074016 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20012
I20260807 15:11:12.086383 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20117
I20260807 15:11:12.099747 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20229
2026-08-07T15:11:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3009 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:12 dist-test-slave-5bn2 krb5kdc[20531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:12 dist-test-slave-5bn2 krb5kdc[20531](info): set up 2 sockets
Aug 07 15:11:12 dist-test-slave-5bn2 krb5kdc[20531](info): commencing operation
krb5kdc: starting...
W20260807 15:11:13.223163 18948 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/test-user.keytab.
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.129.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
Entry for principal kudu/127.18.129.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab.
I20260807 15:11:13.345705 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:35933
--webserver_interface=127.18.129.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:35933,127.18.129.61:35617,127.18.129.60:41787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:13.427281 20549 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:11:13.427529 20549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:13.427559 20549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:13.427584 20549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:13.428714 20549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:13.428754 20549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:13.428772 20549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:13.428788 20549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:13.428803 20549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:13.430325 20549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:35933,127.18.129.61:35617,127.18.129.60:41787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.62.keytab
--principal=kudu/127.18.129.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.62:35933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.129.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:13.430585 20549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:13.430792 20549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:13.435570 20555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:13.435673 20557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:13.435767 20549 server_base.cc:1034] running on GCE node
W20260807 15:11:13.435819 20561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:13.436061 20549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:13.436347 20549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:13.437489 20549 hybrid_clock.cc:648] HybridClock initialized: now 1786115473437448 us; error 58 us; skew 500 ppm
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:13.440042 20549 init.cc:377] Logged in from keytab as kudu/127.18.129.62@KRBTEST.COM (short username kudu)
I20260807 15:11:13.440478 20549 webserver.cc:460] Webserver started at http://127.18.129.62:46627/ using document root <none> and password file <none>
I20260807 15:11:13.440692 20549 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:13.440740 20549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:13.440866 20549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:13.441905 20549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/data/instance:
uuid: "385aa422042241b68cff0415f99935e4"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "3fccecc0ded85a38e6d47bae0f010146"
server_key_iv: "8b34a1b6a7d134ed07491f945e18705f"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.442250 20549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/wal/instance:
uuid: "385aa422042241b68cff0415f99935e4"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "3fccecc0ded85a38e6d47bae0f010146"
server_key_iv: "8b34a1b6a7d134ed07491f945e18705f"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.443665 20549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:13.444499 20572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:13.444667 20549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:13.444756 20549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/wal
uuid: "385aa422042241b68cff0415f99935e4"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "3fccecc0ded85a38e6d47bae0f010146"
server_key_iv: "8b34a1b6a7d134ed07491f945e18705f"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.444815 20549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:13.457167 20549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:13.458971 20549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:13.459115 20549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:13.463060 20549 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.62:35933
I20260807 15:11:13.463109 20657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.62:35933 every 8 connection(s)
I20260807 15:11:13.463428 20549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:13.464413 20658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:13.466006 20658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:13.470793 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20549
I20260807 15:11:13.470921 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-0/wal/instance
I20260807 15:11:13.471113 18948 external_mini_cluster.cc:1428] Setting key 15e6c6eaf4f27012ccfe5184252b2b6c
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:11:13.475359 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.466677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35933 (local address 127.0.0.1:39830)
0807 15:11:13.466925 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.466970 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.467452 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.467684 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.467686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.467942 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.468142 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.468147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.469131 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.469132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.469807 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.469809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.470286 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.470287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.470344 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.470984 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.470994 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.472366 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.473990 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.473996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.473998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.474194 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.474445 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.474445 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.474445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.474518 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.474924 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.474928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.474986 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.475057 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.475189 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":103,"threads_started":1}
I20260807 15:11:13.475401 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.466748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39830 (local address 127.18.129.62:35933)
0807 15:11:13.466956 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.466964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.467057 (+ 93us) server_negotiation.cc:407] Connection header received
0807 15:11:13.467568 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.467570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.467595 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.467670 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.468226 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.469052 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.469870 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.470195 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.470267 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.472447 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.472456 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.472457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.472464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.473907 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.474275 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.474278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.474278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.474386 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.474566 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.474567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.474567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.474837 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.474947 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.475109 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.475288 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":181,"thread_start_us":63,"threads_started":1}
I20260807 15:11:13.476279 20658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:13.476636 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.476500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34282)
0807 15:11:13.476551 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.476609 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 15:11:13.476783 20575 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
I20260807 15:11:13.477206 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.477107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34296)
0807 15:11:13.477152 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.477183 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:11:13.477306 20658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:35617: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.129.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab.
Entry for principal kudu/127.18.129.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab.
I20260807 15:11:13.497563 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.61:35617
--webserver_interface=127.18.129.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab
--principal=kudu/127.18.129.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:35933,127.18.129.61:35617,127.18.129.60:41787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:11:13.526561 20658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } attempt: 1
I20260807 15:11:13.527122 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.526972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34310)
0807 15:11:13.527034 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.527080 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:11:13.527444 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.527355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34322)
0807 15:11:13.527401 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.527426 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 15:11:13.527540 20658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:35617: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
W20260807 15:11:13.582377 20668 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:11:13.582590 20668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:13.582607 20668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:13.582624 20668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:13.583778 20668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:13.583824 20668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:13.583837 20668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:13.583847 20668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:13.583871 20668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:13.585296 20668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:35933,127.18.129.61:35617,127.18.129.60:41787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.61.keytab
--principal=kudu/127.18.129.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.61:35617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.129.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:13.585495 20668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:13.585675 20668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:11:13.590659 20668 server_base.cc:1034] running on GCE node
W20260807 15:11:13.590576 20676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:13.590590 20673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:13.590613 20678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:13.591060 20668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:13.591336 20668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:13.592484 20668 hybrid_clock.cc:648] HybridClock initialized: now 1786115473592471 us; error 31 us; skew 500 ppm
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:13.594998 20668 init.cc:377] Logged in from keytab as kudu/127.18.129.61@KRBTEST.COM (short username kudu)
I20260807 15:11:13.595422 20668 webserver.cc:460] Webserver started at http://127.18.129.61:46811/ using document root <none> and password file <none>
I20260807 15:11:13.595624 20668 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:13.595671 20668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:13.595777 20668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:13.595714 20658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } attempt: 2
I20260807 15:11:13.596154 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.596006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34334)
0807 15:11:13.596074 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.596115 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 15:11:13.596508 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.596415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34338)
0807 15:11:13.596458 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.596486 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 15:11:13.596601 20658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.61:35617: Network error: Client connection negotiation failed: client connection to 127.18.129.61:35617: connect: Connection refused (error 111)
I20260807 15:11:13.597011 20668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/data/instance:
uuid: "c419b03ddbea4f9187c10dcbad7d91e6"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "e5f3ae290a8ad19239b49670e9b47f60"
server_key_iv: "77380da42f1edc25a86a5d3cab17f58d"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.597333 20668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/wal/instance:
uuid: "c419b03ddbea4f9187c10dcbad7d91e6"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "e5f3ae290a8ad19239b49670e9b47f60"
server_key_iv: "77380da42f1edc25a86a5d3cab17f58d"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.598742 20668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:11:13.599627 20690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:13.599799 20668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:13.599890 20668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/wal
uuid: "c419b03ddbea4f9187c10dcbad7d91e6"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "e5f3ae290a8ad19239b49670e9b47f60"
server_key_iv: "77380da42f1edc25a86a5d3cab17f58d"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.599948 20668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:13.612775 20668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:13.614626 20668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:13.614735 20668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:13.618631 20668 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.61:35617
I20260807 15:11:13.618673 20759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.61:35617 every 8 connection(s)
I20260807 15:11:13.619031 20668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/data/info.pb
I20260807 15:11:13.620023 20761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:13.621610 20761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:13.622488 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20668
I20260807 15:11:13.622625 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-1/wal/instance
I20260807 15:11:13.622799 18948 external_mini_cluster.cc:1428] Setting key cfd9840320a0fbb8139ebc5ac39e554a
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.129.60@KRBTEST.COM; defaulting to no policy
I20260807 15:11:13.630642 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.622322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39834 (local address 127.18.129.62:35933)
0807 15:11:13.622377 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.622379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.622570 (+ 191us) server_negotiation.cc:407] Connection header received
0807 15:11:13.623182 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.623184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.623200 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.623246 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.623884 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.624472 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.625450 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.625674 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.625729 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.628036 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.628038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.628038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.628047 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.629336 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.629720 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.629721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.629722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.629778 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.629995 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.629996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.629996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.630129 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.630188 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.630459 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.630550 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:11:13.630820 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.622279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35933 (local address 127.0.0.1:39834)
0807 15:11:13.622451 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.622488 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.623081 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.623327 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.623331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.623583 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.623793 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.623797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.624585 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.624587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.625369 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.625372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.625777 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.625778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.625926 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.626415 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.626425 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.627947 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.629415 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.629422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.629425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.629653 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.629834 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.629835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.629835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.629940 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.630190 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.630198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.630337 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.630428 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.630595 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260807 15:11:13.631356 20761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.129.60@KRBTEST.COM" created.
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:11:13.638414 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.631581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34350)
0807 15:11:13.631748 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.631858 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.631901 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.632006 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.632007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.632118 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.632166 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.632167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.633037 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.633038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.633579 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.633582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.634042 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.634043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.634103 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.634471 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.634475 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.635734 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.637479 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.637479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.637480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.637639 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.637854 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.637855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.637855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.637907 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.638197 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.638197 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.638232 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.638276 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.638351 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 15:11:13.638533 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.631632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.18.129.61:35617)
0807 15:11:13.631783 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.631787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.631910 (+ 123us) server_negotiation.cc:407] Connection header received
0807 15:11:13.631933 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.631934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.631945 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.632007 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.632227 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.632973 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.633676 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.633975 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.634042 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.635809 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.635818 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.635818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.635825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.637400 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.637718 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.637722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.637722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.637787 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.637956 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.637956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.637957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.638138 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.638224 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.638311 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.638415 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260807 15:11:13.639146 20761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:13.639473 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.639365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37890)
0807 15:11:13.639408 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.639453 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 15:11:13.639581 20694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
I20260807 15:11:13.639956 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.639874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37906)
0807 15:11:13.639910 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.639934 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 15:11:13.640044 20761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:41787: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.129.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab.
Entry for principal kudu/127.18.129.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab.
I20260807 15:11:13.649462 18948 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDKyZDM/build/release/bin/kudu
/tmp/dist-test-taskDKyZDM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.60:41787
--webserver_interface=127.18.129.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab
--principal=kudu/127.18.129.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.129.20:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.129.62:35933,127.18.129.61:35617,127.18.129.60:41787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:11:13.687762 20658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } attempt: 3
I20260807 15:11:13.689254 20761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } attempt: 1
I20260807 15:11:13.689677 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.689535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37918)
0807 15:11:13.689590 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.689634 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:11:13.690045 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.689927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37920)
0807 15:11:13.689974 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.690014 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:11:13.690191 20761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:41787: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:11:13.694944 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.688163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34356)
0807 15:11:13.688225 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.688237 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.688285 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.688422 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.688423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.688593 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.688660 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.688662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.689328 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.689329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.690061 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.690064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.690521 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.690522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.690592 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.691055 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.691060 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.692496 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.693995 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.693996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.693996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.694163 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.694331 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.694332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.694332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.694398 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.694609 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.694610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.694658 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.694717 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.694818 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:11:13.694943 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.688238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.18.129.61:35617)
0807 15:11:13.688308 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.688310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.688322 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:11:13.688352 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.688352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.688362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.688410 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.688739 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.689251 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.690165 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.690450 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.690508 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.692578 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.692579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.692580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.692589 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.693919 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.694225 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.694226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.694226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.694277 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.694448 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.694448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.694449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.694554 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.694611 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.694745 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.694818 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:11:13.695325 20658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:13.695658 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.695569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37922)
0807 15:11:13.695611 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.695639 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:11:13.696955 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.695949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37930)
0807 15:11:13.696899 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.696926 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":935}
W20260807 15:11:13.697050 20658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:41787: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
I20260807 15:11:13.728207 20658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } attempt: 1
I20260807 15:11:13.728755 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.728606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37940)
0807 15:11:13.728676 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.728722 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 15:11:13.729125 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.729033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37950)
0807 15:11:13.729078 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.729104 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:11:13.729219 20658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:41787: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
W20260807 15:11:13.733119 20771 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:11:13.733326 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:13.733351 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:13.733387 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:13.734544 20771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:13.734581 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:13.734599 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:13.734615 20771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:13.734638 20771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:13.736152 20771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.129.20:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.129.62:35933,127.18.129.61:35617,127.18.129.60:41787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/krb5kdc/kudu_127.18.129.60.keytab
--principal=kudu/127.18.129.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.129.60:41787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.129.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:13.736392 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:13.736601 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:13.741390 20783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:13.741485 20771 server_base.cc:1034] running on GCE node
W20260807 15:11:13.741402 20777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:13.741405 20779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:13.741788 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:13.742084 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:13.743259 20771 hybrid_clock.cc:648] HybridClock initialized: now 1786115473743245 us; error 24 us; skew 500 ppm
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:13.745747 20771 init.cc:377] Logged in from keytab as kudu/127.18.129.60@KRBTEST.COM (short username kudu)
I20260807 15:11:13.746166 20771 webserver.cc:460] Webserver started at http://127.18.129.60:44879/ using document root <none> and password file <none>
I20260807 15:11:13.746359 20771 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:13.746409 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:13.746532 20771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:13.747725 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/data/instance:
uuid: "99934159659746b29bed3b58164b1b19"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "bbf960673f42fcf2944d19f45588cc04"
server_key_iv: "7e5215a82cf44cab97e3eb29e4e4017d"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.748109 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/wal/instance:
uuid: "99934159659746b29bed3b58164b1b19"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "bbf960673f42fcf2944d19f45588cc04"
server_key_iv: "7e5215a82cf44cab97e3eb29e4e4017d"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.749434 20771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:11:13.750077 20794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:13.750239 20771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:13.750324 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/data,/tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/wal
uuid: "99934159659746b29bed3b58164b1b19"
format_stamp: "Formatted at 2026-08-07 15:11:13 on dist-test-slave-5bn2"
server_key: "bbf960673f42fcf2944d19f45588cc04"
server_key_iv: "7e5215a82cf44cab97e3eb29e4e4017d"
server_key_version: "encryptionkey@0"
I20260807 15:11:13.750385 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:13.758322 20761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } attempt: 2
I20260807 15:11:13.758713 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.758601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37964)
0807 15:11:13.758652 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.758683 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 15:11:13.759075 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.758970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:37972)
0807 15:11:13.759030 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.759055 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 15:11:13.759150 20761 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.129.60:41787: Network error: Client connection negotiation failed: client connection to 127.18.129.60:41787: connect: Connection refused (error 111)
I20260807 15:11:13.764222 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:13.766311 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:13.766449 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:13.770527 20771 rpc_server.cc:307] RPC server started. Bound to: 127.18.129.60:41787
I20260807 15:11:13.770612 20878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.129.60:41787 every 8 connection(s)
I20260807 15:11:13.771000 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/data/info.pb
I20260807 15:11:13.771704 20880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:13.773248 20880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:13.774893 18948 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDKyZDM/build/release/bin/kudu as pid 20771
I20260807 15:11:13.775014 18948 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDKyZDM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115197351496-18948-0/minicluster-data/master-2/wal/instance
I20260807 15:11:13.775210 18948 external_mini_cluster.cc:1428] Setting key 91d34a4d1568d6d8be6733de7fa2e62e
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:11:13.782615 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.773984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.18.129.62:35933)
0807 15:11:13.774043 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.774045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.774260 (+ 215us) server_negotiation.cc:407] Connection header received
0807 15:11:13.774798 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.774800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.774810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.774914 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.775716 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.776196 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.777153 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.777476 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.777529 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.779922 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.779924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.779924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.779931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.781280 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.781679 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.781681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.781682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.781748 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.781983 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.781985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.781986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.782122 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.782183 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.782436 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.782521 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:11:13.782807 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.773953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.62:35933 (local address 127.0.0.1:39840)
0807 15:11:13.774141 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.774180 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.774665 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.775003 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.775008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.775359 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.775600 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.775607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.776281 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.776282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.777072 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.777074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.777574 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.777575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.777715 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.778211 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.778220 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.779819 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.781373 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.781380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.781382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.781618 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.781814 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.781814 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.781814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.781906 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.782185 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.782192 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.782317 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.782404 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.782589 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260807 15:11:13.783365 20880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:13.783948 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.775572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.18.129.60:41787)
0807 15:11:13.775732 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.775737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.775761 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:11:13.775806 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.775808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.775829 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.775910 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.776503 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.777285 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.778249 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.778593 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.778646 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.781178 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.781187 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.781187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.781195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.782751 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.783148 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.783150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.783151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.783214 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.783388 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.783389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.783389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.783542 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.783663 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.783763 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.783842 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.62@KRBTEST.COM
I20260807 15:11:13.789667 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.783607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.61:35617 (local address 127.0.0.1:34368)
0807 15:11:13.783728 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.783738 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.783776 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.783895 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.783896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.784030 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.784066 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.784067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.784775 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.784776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.785323 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.785325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.785650 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.785651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.785705 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.786063 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.786067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.787220 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.788771 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.788772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.788772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.788917 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.789110 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.789111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.789111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.789173 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.789429 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.789430 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.789476 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.789536 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.789606 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 15:11:13.789790 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.783685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.18.129.61:35617)
0807 15:11:13.783733 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.783735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.783766 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:11:13.783825 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.783826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.783836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.783891 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.784124 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.784689 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.785401 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.785595 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.785643 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.787326 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.787327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.787328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.787334 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.788689 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.788982 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.788982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.788983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.789040 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.789229 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.789230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.789230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.789358 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.789425 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.789625 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.789700 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:11:13.790134 20880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:13.791647 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.784663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.18.129.62:35933)
0807 15:11:13.784700 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.784701 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.784753 (+ 52us) server_negotiation.cc:407] Connection header received
0807 15:11:13.784810 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.784811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.784820 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.784875 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.785256 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.785794 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.786586 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.786760 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.786809 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.789088 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.789089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.789090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.789096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.790563 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.790864 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.790865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.790865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.790918 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.791132 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.791132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.791133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.791236 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.791389 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.791423 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.791485 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.60@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.129.61@KRBTEST.COM
I20260807 15:11:13.796430 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.790379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:38002)
0807 15:11:13.790411 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.790420 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.790461 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.790566 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.790567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.790684 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.790709 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.790710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.791340 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.791341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.791828 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.791830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.792299 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.792300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.792346 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.792662 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.792667 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.793961 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.795469 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.795470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.795471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.795671 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.795852 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.795853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.795853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.795918 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.796128 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.796129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.796165 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.796225 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.796319 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260807 15:11:13.796443 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.790410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.18.129.60:41787)
0807 15:11:13.790458 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.790459 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.790468 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:13.790505 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.790506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.790515 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.790555 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.790764 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.791273 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.791905 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.792199 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.792257 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.794052 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.794053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.794053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.794060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.795340 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.795746 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.795747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.795747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.795796 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.795972 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.795973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.795973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.796075 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.796129 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.796250 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.796363 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":33}
I20260807 15:11:13.798028 20880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19: Bootstrap starting.
I20260807 15:11:13.798979 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.792248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34380 (local address 127.18.129.61:35617)
0807 15:11:13.792292 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.792294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.792301 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:13.792319 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.792320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.792328 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.792369 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.792896 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.793479 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.794263 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.794478 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.794530 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.796829 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.796830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.796831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.796838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.797972 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.798280 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.798281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.798282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.798336 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.798501 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.798502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.798502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.798612 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.798760 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.798795 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.798897 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:11:13.799005 20880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:13.799309 20880 log.cc:824] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:13.800148 20880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19: No bootstrap required, opened a new log
I20260807 15:11:13.800465 18948 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:13.801755 20880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } }
I20260807 15:11:13.801887 20880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99934159659746b29bed3b58164b1b19, State: Initialized, Role: FOLLOWER
I20260807 15:11:13.801967 20880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } }
I20260807 15:11:13.801432 20618 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "385aa422042241b68cff0415f99935e4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39842
I20260807 15:11:13.802286 20880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:13.802245 20891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.802336 20891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:11:13.804171 20903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 15:11:13.804327 20658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } attempt: 2
W20260807 15:11:13.804456 20903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:13.804512 20771 master_runner.cc:186] Error getting master registration for 127.18.129.62:35933: OK, instance_id { permanent_uuid: "385aa422042241b68cff0415f99935e4" instance_seqno: 1786115473461955 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:11:13.807423 20771 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.62@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:11:13.810739 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.804607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.18.129.60:41787)
0807 15:11:13.804639 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.804641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.804664 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:11:13.804721 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.804721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.804730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.804774 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.805026 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.805472 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.806112 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.806306 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.806358 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.808246 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.808248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.808249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.808256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.809519 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.809945 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.809946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.809947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.810001 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.810207 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.810208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.810208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.810360 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.810421 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.810573 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.810647 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 15:11:13.810745 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.804533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:38008)
0807 15:11:13.804620 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.804631 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.804678 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.804776 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.804778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.804923 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.804962 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.804964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.805537 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.805537 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.806043 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.806045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.806361 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.806362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.806419 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.806797 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.806802 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.808162 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.809588 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.809589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.809590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.809827 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.810066 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.810067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.810067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.810149 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.810424 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.810425 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.810476 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.810549 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.810649 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 15:11:13.812376 20658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4: Bootstrap starting.
I20260807 15:11:13.813098 20658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:13.813405 20658 log.cc:824] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:13.814111 20658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4: No bootstrap required, opened a new log
I20260807 15:11:13.815148 20658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } }
I20260807 15:11:13.815260 20658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385aa422042241b68cff0415f99935e4, State: Initialized, Role: FOLLOWER
I20260807 15:11:13.815333 20658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } }
I20260807 15:11:13.815589 20910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.815685 20910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:13.815701 20618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:11:13.815745 20618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:13.816324 20618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } }
I20260807 15:11:13.816565 20618 leader_election.cc:288] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [CANDIDATE]: Term 1 election: Requested vote from peers c419b03ddbea4f9187c10dcbad7d91e6 (127.18.129.61:35617), 99934159659746b29bed3b58164b1b19 (127.18.129.60:41787)
I20260807 15:11:13.817803 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "385aa422042241b68cff0415f99935e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "99934159659746b29bed3b58164b1b19"
I20260807 15:11:13.817932 20840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:13.818877 20840 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 385aa422042241b68cff0415f99935e4 in term 1.
I20260807 15:11:13.815631 20658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:13.819897 20583 leader_election.cc:302] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 385aa422042241b68cff0415f99935e4, 99934159659746b29bed3b58164b1b19; no voters:
I20260807 15:11:13.820045 20910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:13.820174 20910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [term 1 LEADER]: Becoming Leader. State: Replica: 385aa422042241b68cff0415f99935e4, State: Running, Role: LEADER
I20260807 15:11:13.820272 20910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } }
I20260807 15:11:13.820695 20910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 385aa422042241b68cff0415f99935e4. Latest consensus state: current_term: 1 leader_uuid: "385aa422042241b68cff0415f99935e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.820770 20910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:13.822103 20549 master_runner.cc:186] Error getting master registration for 127.18.129.61:35617: OK, instance_id { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" instance_seqno: 1786115473617562 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:11:13.825426 20549 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:11:13.825659 20950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:13.827224 20950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:13.827831 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.817857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.18.129.62:35933)
0807 15:11:13.817895 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.817897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.818151 (+ 254us) server_negotiation.cc:407] Connection header received
0807 15:11:13.818244 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.818245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.818255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.818282 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.819069 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.819776 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.821961 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.822218 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.822258 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.823832 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.823834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.823834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.823842 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.825238 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.825797 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.825798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.825798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.825858 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.826057 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.826059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.826059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.826199 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.826290 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.827664 (+ 1374us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.827741 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:11:13.829306 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.817927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.18.129.60:41787)
0807 15:11:13.817987 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.817989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.819103 (+ 1114us) server_negotiation.cc:407] Connection header received
0807 15:11:13.819285 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.819286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.819295 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.819322 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.820724 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.821442 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.823068 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.823384 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.823422 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.825662 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.825664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.825664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.825684 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.827514 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.828527 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.828528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.828528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.828586 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.828771 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.828772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.828773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.828888 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.828954 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.829143 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.829217 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:11:13.831049 20840 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [term 1 FOLLOWER]: Refusing update from remote peer 385aa422042241b68cff0415f99935e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:13.831416 20941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:13.832908 20891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 385aa422042241b68cff0415f99935e4. Latest consensus state: current_term: 1 leader_uuid: "385aa422042241b68cff0415f99935e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.833001 20891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:13.833284 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.820591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.18.129.61:35617)
0807 15:11:13.820634 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.820636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.820859 (+ 223us) server_negotiation.cc:407] Connection header received
0807 15:11:13.820956 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.820958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.820969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.821000 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.826632 (+ 5632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.827307 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.828708 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.828970 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.829031 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.830279 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.830281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.830281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.830288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.831823 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.832181 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.832182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.832183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.832239 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.832437 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.832437 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.832438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.832583 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.832658 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.833107 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.833190 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:11:13.847486 20910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "385aa422042241b68cff0415f99935e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.847612 20910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:13.847681 20954 mvcc.cc:205] Tried to move back new op lower bound from 7315928980803158016 to 7315928980768440320. Current Snapshot: MvccSnapshot[applied={T|T < 7315928980803158016}]
I20260807 15:11:13.848166 20891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "385aa422042241b68cff0415f99935e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.848258 20891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:13.848440 20957 mvcc.cc:205] Tried to move back new op lower bound from 7315928980803158016 to 7315928980768440320. Current Snapshot: MvccSnapshot[applied={T|T < 7315928980803158016}]
I20260807 15:11:13.848475 20950 catalog_manager.cc:1347] Generated new cluster ID: 5ac994256d3a4bffa3d98c561959486a
I20260807 15:11:13.848527 20950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:13.850267 20761 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } attempt: 3
Aug 07 15:11:13 dist-test-slave-5bn2 krb5kdc[20531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115473, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.129.61@KRBTEST.COM for kudu/127.18.129.60@KRBTEST.COM
I20260807 15:11:13.860632 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.850520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.129.60:41787 (local address 127.0.0.1:38026)
0807 15:11:13.850568 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:13.850579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:13.850619 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:13.852899 (+ 2280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:13.852900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:13.853091 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:13.853140 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.853142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.854022 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.854023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.854880 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:13.854882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.855494 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.855496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:13.855575 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.856046 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:13.856052 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:13.857596 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:13.859430 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.859431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.859431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.859648 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.859893 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:13.859894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:13.859895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:13.859972 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:13.860298 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:13.860300 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:13.860360 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 15:11:13.860427 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:11:13.860528 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260807 15:11:13.861120 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:13.852728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.18.129.60:41787)
0807 15:11:13.852779 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.852781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.852788 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:13.852809 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.852810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.852820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.852880 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.853210 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.853958 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.854952 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:13.855252 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:13.855316 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:13.857696 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:13.857698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:13.857698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:13.857705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:13.859315 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.859736 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.859737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.859738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.859812 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:13.860052 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:13.860053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:13.860053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:13.860197 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:13.860256 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:13.860954 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 15:11:13.861031 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:11:13.862957 20761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6: Bootstrap starting.
I20260807 15:11:13.863719 20761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:13.864009 20761 log.cc:824] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:13.864957 20761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6: No bootstrap required, opened a new log
I20260807 15:11:13.867010 20761 raft_consensus.cc:348] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } }
I20260807 15:11:13.867157 20761 raft_consensus.cc:729] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c419b03ddbea4f9187c10dcbad7d91e6, State: Initialized, Role: FOLLOWER
I20260807 15:11:13.867240 20761 consensus_queue.cc:260] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } }
I20260807 15:11:13.867564 20761 sys_catalog.cc:564] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:13.867654 20723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:13.868548 20723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [term 1 FOLLOWER]: Refusing update from remote peer 385aa422042241b68cff0415f99935e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:13.868858 20941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:13.867663 20724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "385aa422042241b68cff0415f99935e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c419b03ddbea4f9187c10dcbad7d91e6"
I20260807 15:11:13.869758 20724 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 385aa422042241b68cff0415f99935e4 in term 1.
I20260807 15:11:13.870044 20962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.870162 20962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:13.872195 20962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 385aa422042241b68cff0415f99935e4. Latest consensus state: current_term: 1 leader_uuid: "385aa422042241b68cff0415f99935e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.872282 20962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:13.872319 20962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "385aa422042241b68cff0415f99935e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.872364 20962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:13.883544 20979 catalog_manager.cc:1259] Loaded cluster ID: 5ac994256d3a4bffa3d98c561959486a
I20260807 15:11:13.883611 20979 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6: loading cluster ID for follower catalog manager: success
W20260807 15:11:13.883741 20979 catalog_manager.cc:1576] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260807 15:11:13.883791 20979 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:13.891100 20941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "385aa422042241b68cff0415f99935e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "385aa422042241b68cff0415f99935e4" member_type: VOTER last_known_addr { host: "127.18.129.62" port: 35933 } } peers { permanent_uuid: "c419b03ddbea4f9187c10dcbad7d91e6" member_type: VOTER last_known_addr { host: "127.18.129.61" port: 35617 } } peers { permanent_uuid: "99934159659746b29bed3b58164b1b19" member_type: VOTER last_known_addr { host: "127.18.129.60" port: 41787 } } }
I20260807 15:11:13.891212 20941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:13.892670 20950 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:13.893440 20950 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:13.902565 20950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 385aa422042241b68cff0415f99935e4: Generated new TSK 0
I20260807 15:11:13.902882 20950 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:11:13.934930 20886 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.933509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38034 (local address 127.18.129.60:41787)
0807 15:11:13.933556 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.933559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.934188 (+ 629us) server_negotiation.cc:407] Connection header received
0807 15:11:13.934292 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.934293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.934304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.934333 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.934830 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38034: BlockingRecv error: recv got EOF from 127.0.0.1:38034 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 15:11:13.935689 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.933911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.18.129.61:35617)
0807 15:11:13.933952 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.933954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.935186 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 15:11:13.935279 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.935280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.935291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.935318 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.935625 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34406: BlockingRecv error: recv got EOF from 127.0.0.1:34406 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:11:13.938916 20661 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.938188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.18.129.62:35933)
0807 15:11:13.938231 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.938232 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.938354 (+ 122us) server_negotiation.cc:407] Connection header received
0807 15:11:13.938441 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.938442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.938452 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.938479 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.938826 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39862: BlockingRecv error: recv got EOF from 127.0.0.1:39862 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 15:11:13.941185 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.940459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.18.129.61:35617)
0807 15:11:13.940503 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.940505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.940679 (+ 174us) server_negotiation.cc:407] Connection header received
0807 15:11:13.940773 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.940774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.940793 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.940820 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.941137 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34408: BlockingRecv error: recv got EOF from 127.0.0.1:34408 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 15:11:13.944164 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.943459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.18.129.61:35617)
0807 15:11:13.943490 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.943492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.943704 (+ 212us) server_negotiation.cc:407] Connection header received
0807 15:11:13.943790 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.943791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.943801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.943826 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.944121 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34418: BlockingRecv error: recv got EOF from 127.0.0.1:34418 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 15:11:13.947997 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.947278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34434 (local address 127.18.129.61:35617)
0807 15:11:13.947311 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.947312 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.947531 (+ 219us) server_negotiation.cc:407] Connection header received
0807 15:11:13.947627 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.947628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.947637 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.947662 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.947954 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34434: BlockingRecv error: recv got EOF from 127.0.0.1:34434 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260807 15:11:13.953894 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.953151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34442 (local address 127.18.129.61:35617)
0807 15:11:13.953184 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.953185 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.953431 (+ 246us) server_negotiation.cc:407] Connection header received
0807 15:11:13.953517 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.953518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.953528 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.953554 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.953850 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34442: BlockingRecv error: recv got EOF from 127.0.0.1:34442 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 15:11:13.963879 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.963098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34456 (local address 127.18.129.61:35617)
0807 15:11:13.963138 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.963139 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.963369 (+ 230us) server_negotiation.cc:407] Connection header received
0807 15:11:13.963461 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.963461 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.963472 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.963498 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.963825 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34456: BlockingRecv error: recv got EOF from 127.0.0.1:34456 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 15:11:13.982195 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:13.981232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34472 (local address 127.18.129.61:35617)
0807 15:11:13.981275 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:13.981277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:13.981562 (+ 285us) server_negotiation.cc:407] Connection header received
0807 15:11:13.981672 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:13.981673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:13.981685 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:13.981716 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:13.982135 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34472: BlockingRecv error: recv got EOF from 127.0.0.1:34472 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:11:14.016407 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:14.015713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.18.129.61:35617)
0807 15:11:14.015765 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.015767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.015847 (+ 80us) server_negotiation.cc:407] Connection header received
0807 15:11:14.015921 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.015922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.015932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.015981 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:14.016340 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34488: BlockingRecv error: recv got EOF from 127.0.0.1:34488 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 15:11:14.082269 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:14.081613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.18.129.61:35617)
0807 15:11:14.081677 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.081679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.081778 (+ 99us) server_negotiation.cc:407] Connection header received
0807 15:11:14.081829 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.081830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.081840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.081901 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:14.082211 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34496: BlockingRecv error: recv got EOF from 127.0.0.1:34496 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 15:11:14.212037 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:14.211409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.18.129.61:35617)
0807 15:11:14.211471 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.211473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.211562 (+ 89us) server_negotiation.cc:407] Connection header received
0807 15:11:14.211622 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.211622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.211633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.211694 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:14.211982 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34510: BlockingRecv error: recv got EOF from 127.0.0.1:34510 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 15:11:14.470036 20767 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:14.469401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.18.129.61:35617)
0807 15:11:14.469468 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.469470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.469514 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:11:14.469567 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.469568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.469579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.469643 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:14.469983 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34516: BlockingRecv error: recv got EOF from 127.0.0.1:34516 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:11:14.804965 20903 catalog_manager.cc:1259] Loaded cluster ID: 5ac994256d3a4bffa3d98c561959486a
I20260807 15:11:14.805048 20903 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19: loading cluster ID for follower catalog manager: success
I20260807 15:11:14.805918 20903 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19: acquiring CA information for follower catalog manager: success
I20260807 15:11:14.806231 20903 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 99934159659746b29bed3b58164b1b19: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:11:14.884902 20979 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6: acquiring CA information for follower catalog manager: success
I20260807 15:11:14.885280 20979 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c419b03ddbea4f9187c10dcbad7d91e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:11:14.985760 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:14.983239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.18.129.61:35617)
0807 15:11:14.983373 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.983376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.983387 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:14.983458 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.983459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.983539 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.983603 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:14.983755 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.984259 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.985002 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.985200 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.985256 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:14.985418 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:14.985538 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:14.985605 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:14.985647 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 15:11:14.985668 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260807 15:11:14.989768 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:14.987437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.18.129.60:41787)
0807 15:11:14.987538 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.987540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.987548 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:14.987601 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.987602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.987638 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.987700 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:14.987776 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.988370 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.989196 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.989374 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.989418 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:14.989511 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:14.989606 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:14.989657 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:14.989672 (+ 15us) server_negotiation.cc:299] Negotiation successful
0807 15:11:14.989688 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
I20260807 15:11:14.994091 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:14.991290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39878 (local address 127.18.129.62:35933)
0807 15:11:14.991395 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.991397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.991494 (+ 97us) server_negotiation.cc:407] Connection header received
0807 15:11:14.991569 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.991570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.991623 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.991691 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:14.991810 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.992547 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.993254 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.993528 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.993588 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:14.993742 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:14.993844 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:14.993907 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:14.993955 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 15:11:14.993982 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":55,"threads_started":1}
I20260807 15:11:14.994138 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:14.991277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.18.129.60:41787)
0807 15:11:14.991376 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.991378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.991635 (+ 257us) server_negotiation.cc:407] Connection header received
0807 15:11:14.991724 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.991725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.991737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.991762 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:14.991910 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.992355 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.993398 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.993673 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.993725 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:14.993903 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:14.993934 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:14.993962 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:14.994062 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 15:11:14.994081 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 15:11:14.994140 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:14.991321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.18.129.61:35617)
0807 15:11:14.991363 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:14.991365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:14.991508 (+ 143us) server_negotiation.cc:407] Connection header received
0807 15:11:14.991559 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:14.991560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:14.991571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:14.991619 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:14.991752 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.992467 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.993421 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:14.993670 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:14.993726 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:14.993862 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:14.993896 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:14.993941 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:14.994028 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 15:11:14.994053 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:11:14.998826 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20549
I20260807 15:11:15.013895 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20668
I20260807 15:11:15.023804 18948 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDKyZDM/build/release/bin/kudu with pid 20771
2026-08-07T15:11:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2923 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5933 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (277683 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST